Repository navigation
feat(heatmaps): add page history models - #114215
trunk-io[bot] merged 2 commits into
Conversation
🤖 CI report
|
| First copy | Second copy | Lines | Tokens |
|---|---|---|---|
ee/hogai/chat_agent/sql/prompts.py:62 |
posthog/hogql/ai.py:65 |
67 | 628 |
products/messaging/backend/tests/api/test_message_preferences.py:33 |
products/messaging/backend/tests/test_message_preferences.py:15 |
26 | 193 |
products/warehouse_sources/backend/temporal/data_imports/sources/gusto/source.py:1 |
products/warehouse_sources/backend/temporal/data_imports/sources/shopify/source.py:1 |
27 | 154 |
products/ai_observability/backend/models/llm_prompt.py:158 |
products/skills/backend/models/skills.py:223 |
14 | 105 |
products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/schemas.py:256 |
products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/schemas.py:560 |
46 | 92 |
products/messaging/backend/presentation/views/message_preferences.py:5 |
products/messaging/backend/presentation/views/message_suppression.py:7 |
12 | 75 |
posthog/api/services/llm_prompt.py:423 |
posthog/api/services/llm_prompt.py:499 |
20 | 70 |
⚠️ Duplication (TypeScript) — 1 new duplicated block (worst 76 tokens)
New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.
| First copy | Second copy | Lines | Tokens |
|---|---|---|---|
products/ai_observability/frontend/prompts/llmPromptsLogic.ts:236 |
products/skills/frontend/llmSkillsLogic.ts:613 |
12 | 76 |
✅ MCP agent API — no agent API changes
This PR no longer changes the tools, params, scopes or annotations agents see.
⚠️ Backend coverage — 99.0% of changed backend lines covered — 1 uncovered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ████████████████████ 99.0% (154 / 155)
| File | Patch | Uncovered changed lines |
|---|---|---|
products/web_analytics/backend/models/heatmap_screenshot_history.py |
98.6% | 53 |
🤖 Agents: add a test only if an uncovered line exposes a realistic regression that existing tests miss. Otherwise explain why no new test is needed under "How did you test this code?". Gap list: the patch-coverage artifact on this run (gh run download 252178111856007 -n patch-coverage), or the coverage-data block at the end of this comment.
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
platform_features |
██░░░░░░░░░░░░░░░░░░ 12.1% |
7 / 58 |
demo |
███████████░░░░░░░░░ 55.9% |
1,510 / 2,700 |
data_tools |
████████████░░░░░░░░ 61.2% |
90 / 147 |
warehouse_sources_queue |
█████████████░░░░░░░ 66.8% |
1,921 / 2,875 |
aeo |
███████████████░░░░░ 76.3% |
617 / 809 |
batch_exports |
████████████████░░░░ 80.1% |
21,343 / 26,648 |
apm |
█████████████████░░░ 84.1% |
1,306 / 1,553 |
engineering_analytics |
█████████████████░░░ 85.3% |
10,670 / 12,514 |
mcp_analytics |
█████████████████░░░ 85.7% |
4,844 / 5,651 |
ml_inference |
█████████████████░░░ 86.3% |
524 / 607 |
warehouse_suggestions |
█████████████████░░░ 87.1% |
2,192 / 2,518 |
posthog_ai |
██████████████████░░ 87.7% |
4,226 / 4,817 |
cdp |
██████████████████░░ 88.0% |
4,656 / 5,290 |
notebooks |
██████████████████░░ 88.4% |
15,201 / 17,196 |
today |
██████████████████░░ 88.6% |
2,866 / 3,233 |
mcp_registry |
██████████████████░░ 89.0% |
1,596 / 1,794 |
product_tours |
██████████████████░░ 89.1% |
1,337 / 1,500 |
signals |
██████████████████░░ 89.5% |
66,073 / 73,797 |
webmcp |
██████████████████░░ 89.6% |
240 / 268 |
cohorts |
██████████████████░░ 89.6% |
8,458 / 9,438 |
dashboards |
██████████████████░░ 89.8% |
7,111 / 7,915 |
data_modeling |
██████████████████░░ 90.1% |
10,831 / 12,021 |
ai_training |
██████████████████░░ 90.4% |
349 / 386 |
tasks |
██████████████████░░ 90.6% |
80,785 / 89,194 |
visual_review |
██████████████████░░ 90.7% |
9,842 / 10,856 |
data_warehouse |
██████████████████░░ 90.7% |
14,656 / 16,155 |
canvas |
██████████████████░░ 90.9% |
7,748 / 8,522 |
exports |
██████████████████░░ 91.0% |
9,698 / 10,663 |
business_knowledge |
██████████████████░░ 91.0% |
8,625 / 9,480 |
autoresearch |
██████████████████░░ 91.2% |
10,009 / 10,971 |
managed_warehouse |
██████████████████░░ 91.2% |
11,053 / 12,114 |
error_tracking |
██████████████████░░ 91.6% |
16,797 / 18,328 |
wizard |
██████████████████░░ 91.8% |
6,008 / 6,548 |
streamlit_apps |
██████████████████░░ 91.8% |
3,087 / 3,362 |
stamphog |
██████████████████░░ 92.0% |
8,242 / 8,963 |
conversations |
██████████████████░░ 92.1% |
29,145 / 31,640 |
managed_migrations |
██████████████████░░ 92.5% |
1,600 / 1,730 |
early_access_features |
███████████████████░ 92.6% |
1,339 / 1,446 |
alerts |
███████████████████░ 92.9% |
7,246 / 7,797 |
surveys |
███████████████████░ 93.1% |
6,626 / 7,118 |
web_analytics |
███████████████████░ 93.1% |
24,096 / 25,877 |
notifications |
███████████████████░ 93.2% |
1,152 / 1,236 |
approvals |
███████████████████░ 93.3% |
4,493 / 4,816 |
review_hog |
███████████████████░ 93.4% |
13,299 / 14,244 |
cross_project_dashboards |
███████████████████░ 93.4% |
880 / 942 |
slack_app |
███████████████████░ 93.5% |
14,869 / 15,897 |
context_layer |
███████████████████░ 93.6% |
3,408 / 3,640 |
marketing_analytics |
███████████████████░ 93.8% |
19,836 / 21,158 |
workflows |
███████████████████░ 93.8% |
16,389 / 17,477 |
billing_alerts |
███████████████████░ 93.9% |
2,090 / 2,226 |
customer_analytics |
███████████████████░ 93.9% |
26,079 / 27,773 |
mcp_store |
███████████████████░ 93.9% |
9,008 / 9,593 |
messaging |
███████████████████░ 94.2% |
4,315 / 4,582 |
experiments |
███████████████████░ 94.2% |
33,570 / 35,647 |
ai_observability |
███████████████████░ 94.2% |
26,434 / 28,069 |
replay_vision |
███████████████████░ 94.3% |
29,002 / 30,756 |
legal_documents |
███████████████████░ 94.3% |
2,289 / 2,427 |
logs |
███████████████████░ 94.5% |
15,850 / 16,770 |
actions |
███████████████████░ 94.6% |
973 / 1,029 |
endpoints |
███████████████████░ 94.8% |
9,298 / 9,812 |
reminders |
███████████████████░ 94.8% |
760 / 802 |
growth |
███████████████████░ 94.8% |
11,268 / 11,880 |
skills |
███████████████████░ 94.9% |
7,024 / 7,403 |
annotations |
███████████████████░ 95.0% |
816 / 859 |
tracing |
███████████████████░ 95.2% |
3,617 / 3,801 |
data_catalog |
███████████████████░ 95.5% |
4,420 / 4,628 |
access_control |
███████████████████░ 95.5% |
7,779 / 8,143 |
product_analytics |
███████████████████░ 95.7% |
28,602 / 29,879 |
alerts_platform |
███████████████████░ 95.8% |
5,012 / 5,234 |
revenue_analytics |
███████████████████░ 95.9% |
1,878 / 1,959 |
data_quality |
███████████████████░ 95.9% |
8,569 / 8,933 |
feature_flags |
███████████████████░ 96.2% |
27,109 / 28,175 |
warehouse_sources |
███████████████████░ 96.4% |
451,249 / 468,151 |
security |
███████████████████░ 96.7% |
1,452 / 1,501 |
pulse |
███████████████████░ 97.4% |
2,023 / 2,078 |
metrics |
████████████████████ 97.8% |
3,975 / 4,063 |
analytics_platform |
████████████████████ 98.0% |
2,775 / 2,833 |
field_notes |
████████████████████ 99.4% |
172 / 173 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
⚠️ MCP snapshots — 2 updated (2 modified, 0 added, 0 deleted)
Snapshots: MCP unit test snapshots updated
Changes: 2 snapshots (2 modified, 0 added, 0 deleted)
What this means:
- Snapshots have been automatically updated to match current output
Next steps:
- Review the changes to ensure they're intentional
- If unexpected, investigate what caused the output to change
⚠️ Django migration SQL — 2 new migrations to review
We've detected new migrations on this PR. Review the SQL output for each migration:
products/web_analytics/backend/migrations/0015_heatmap_page_history.py
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/anyio/from_thread.py:119: SyntaxWarning: 'return' in a 'finally' block
return result
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/structlog/stdlib.py:1166: UserWarning: Remove `format_exc_info` from your processor chain if you want pretty exceptions.
ed = p(logger, meth_name, ed) # type: ignore[arg-type]
2026-10-09T19:01:19.979750Z [error ] Path must be a valid database or directory containing databases. [posthog.exceptions_capture] pid=5899 tid=140541971090304
Traceback (most recent call last):
File "/home/runner/work/posthog/posthog/posthog/geoip.py", line 15, in <module>
geoip: Optional[GeoIP2] = GeoIP2(cache=8)
~~~~~~^^^^^^^^^
File "/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/django/contrib/gis/geoip2.py", line 116, in __init__
raise GeoIP2Exception(
"Path must be a valid database or directory containing databases."
)
django.contrib.gis.geoip2.GeoIP2Exception: Path must be a valid database or directory containing databases.
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/sshtunnel.py:1040: SyntaxWarning: 'return' in a 'finally' block
return (ssh_host,
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/langchain_core/_api/deprecation.py:27: UserWarning: Core Pydantic V1 functionality isn't compatible with Python 3.14 or greater.
from pydantic.v1.fields import FieldInfo as FieldInfoV1
System check identified some issues:
WARNINGS:
?: (axes.W001) You are using the django-axes cache handler for login attempt tracking. Your cache configuration is however invalid and will not work correctly with django-axes. This can leave security holes in your login systems as attempts are not tracked correctly. Reconfigure settings.AXES_CACHE and settings.CACHES per django-axes configuration documentation.
?: (staticfiles.W004) The directory '/home/runner/work/posthog/posthog/frontend/dist' in the STATICFILES_DIRS setting does not exist.
BEGIN;
--
-- Create model HeatmapScreenshotHistory
--
CREATE TABLE "posthog_heatmapscreenshothistory" ("id" uuid NOT NULL PRIMARY KEY, "width" integer NOT NULL, "captured_on" date NOT NULL, "captured_at" timestamp with time zone NULL, "timezone" varchar(100) NOT NULL, "day_start" timestamp with time zone NOT NULL, "day_end" timestamp with time zone NOT NULL, "expires_at" timestamp with time zone NOT NULL, "revision" uuid NULL, "has_thumbnail" boolean NOT NULL, "trigger" varchar(20) NOT NULL, "status" varchar(20) NOT NULL, "failure_cause" varchar(100) NULL, "page_status" integer NULL, "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL, "heatmap_id" uuid NOT NULL, "team_id" integer NOT NULL, CONSTRAINT "heatmap_history_one_per_day" UNIQUE ("heatmap_id", "captured_on"));
--
-- Create model HeatmapCaptureRequest
--
CREATE TABLE "posthog_heatmapcapturerequest" ("id" uuid NOT NULL PRIMARY KEY, "state" varchar(16) NOT NULL, "trigger" varchar(20) NOT NULL, "captured_on" date NOT NULL, "timezone" varchar(100) NOT NULL, "day_start" timestamp with time zone NOT NULL, "day_end" timestamp with time zone NOT NULL, "width" integer NOT NULL CHECK ("width" >= 0), "url" varchar(2000) NOT NULL, "block_consent_modals" boolean NOT NULL, "input_signature" varchar(64) NOT NULL, "claim_id" uuid NULL, "deadline" timestamp with time zone NOT NULL, "expires_at" timestamp with time zone NOT NULL, "failure_cause" varchar(100) NULL, "page_status" integer NULL, "created_at" timestamp with time zone NOT NULL, "completed_at" timestamp with time zone NULL, "heatmap_id" uuid NOT NULL, "team_id" integer NOT NULL, "history_id" uuid NOT NULL);
--
-- Add field last_viewed_at to savedheatmap
--
ALTER TABLE "posthog_heatmapsaved" ADD COLUMN "last_viewed_at" timestamp with time zone NULL;
--
-- Add field next_history_capture_at to savedheatmap
--
ALTER TABLE "posthog_heatmapsaved" ADD COLUMN "next_history_capture_at" timestamp with time zone NULL;
--
-- Add field history_configuration_revision to savedheatmap
--
ALTER TABLE "posthog_heatmapsaved" ADD COLUMN "history_configuration_revision" uuid NULL;
--
-- Add field latest_request to heatmapscreenshothistory
--
ALTER TABLE "posthog_heatmapscreenshothistory" ADD COLUMN "latest_request_id" uuid NULL CONSTRAINT "posthog_heatmapscree_latest_request_id_cb7a3223_fk_posthog_h" REFERENCES "posthog_heatmapcapturerequest"("id") DEFERRABLE INITIALLY DEFERRED; SET CONSTRAINTS "posthog_heatmapscree_latest_request_id_cb7a3223_fk_posthog_h" IMMEDIATE;
ALTER TABLE "posthog_heatmapscreenshothistory" ADD CONSTRAINT "posthog_heatmapscree_heatmap_id_a9db7d82_fk_posthog_h" FOREIGN KEY ("heatmap_id") REFERENCES "posthog_heatmapsaved" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_heatmapscreenshothistory_expires_at_7611b94e" ON "posthog_heatmapscreenshothistory" ("expires_at");
CREATE INDEX "posthog_heatmapscreenshothistory_heatmap_id_a9db7d82" ON "posthog_heatmapscreenshothistory" ("heatmap_id");
CREATE INDEX "posthog_heatmapscreenshothistory_team_id_01764dc9" ON "posthog_heatmapscreenshothistory" ("team_id");
ALTER TABLE "posthog_heatmapcapturerequest" ADD CONSTRAINT "posthog_heatmapcaptu_heatmap_id_7220da2f_fk_posthog_h" FOREIGN KEY ("heatmap_id") REFERENCES "posthog_heatmapsaved" ("id") DEFERRABLE INITIALLY DEFERRED;
ALTER TABLE "posthog_heatmapcapturerequest" ADD CONSTRAINT "posthog_heatmapcaptu_history_id_ee8028a4_fk_posthog_h" FOREIGN KEY ("history_id") REFERENCES "posthog_heatmapscreenshothistory" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE UNIQUE INDEX "heatmap_history_one_scheduled" ON "posthog_heatmapcapturerequest" ("heatmap_id", "captured_on") WHERE "trigger" = 'scheduled';
CREATE INDEX "posthog_heatmapcapturerequest_expires_at_3d66d835" ON "posthog_heatmapcapturerequest" ("expires_at");
CREATE INDEX "posthog_heatmapcapturerequest_heatmap_id_7220da2f" ON "posthog_heatmapcapturerequest" ("heatmap_id");
CREATE INDEX "posthog_heatmapcapturerequest_team_id_a6aa01ab" ON "posthog_heatmapcapturerequest" ("team_id");
CREATE INDEX "posthog_heatmapcapturerequest_history_id_ee8028a4" ON "posthog_heatmapcapturerequest" ("history_id");
CREATE INDEX "heatmap_request_global_idx" ON "posthog_heatmapcapturerequest" ("trigger", "created_at");
CREATE INDEX "heatmap_request_team_idx" ON "posthog_heatmapcapturerequest" ("team_id", "captured_on", "trigger");
CREATE INDEX "heatmap_request_live_idx" ON "posthog_heatmapcapturerequest" ("deadline") WHERE "state" IN ('queued', 'running');
CREATE INDEX "posthog_heatmapscreenshothistory_latest_request_id_cb7a3223" ON "posthog_heatmapscreenshothistory" ("latest_request_id");
COMMIT;products/web_analytics/backend/migrations/0016_savedheatmap_last_viewed_idx.py
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/structlog/stdlib.py:1166: UserWarning: Remove `format_exc_info` from your processor chain if you want pretty exceptions.
ed = p(logger, meth_name, ed) # type: ignore[arg-type]
2026-10-09T19:02:09.147597Z [error ] Path must be a valid database or directory containing databases. [posthog.exceptions_capture] pid=7150 tid=139670663805824
Traceback (most recent call last):
File "/home/runner/work/posthog/posthog/posthog/geoip.py", line 15, in <module>
geoip: Optional[GeoIP2] = GeoIP2(cache=8)
~~~~~~^^^^^^^^^
File "/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/django/contrib/gis/geoip2.py", line 116, in __init__
raise GeoIP2Exception(
"Path must be a valid database or directory containing databases."
)
django.contrib.gis.geoip2.GeoIP2Exception: Path must be a valid database or directory containing databases.
/home/runner/work/tmp/tool_cache/Python/3.14.7/x64/lib/python3.14/site-packages/langchain_core/_api/deprecation.py:27: UserWarning: Core Pydantic V1 functionality isn't compatible with Python 3.14 or greater.
from pydantic.v1.fields import FieldInfo as FieldInfoV1
System check identified some issues:
WARNINGS:
?: (axes.W001) You are using the django-axes cache handler for login attempt tracking. Your cache configuration is however invalid and will not work correctly with django-axes. This can leave security holes in your login systems as attempts are not tracked correctly. Reconfigure settings.AXES_CACHE and settings.CACHES per django-axes configuration documentation.
?: (staticfiles.W004) The directory '/home/runner/work/posthog/posthog/frontend/dist' in the STATICFILES_DIRS setting does not exist.
--
-- Concurrently create index heatmap_last_viewed_idx on field(s) last_viewed_at of model savedheatmap
--
SET lock_timeout = 0;
SET statement_timeout = 0;
CREATE INDEX CONCURRENTLY "heatmap_last_viewed_idx" ON "posthog_heatmapsaved" ("last_viewed_at");Last updated: 2026-10-09 19:02 UTC (45a60a5)
✅ Django migration risk — migration analysis complete
We've analyzed your migrations for potential risks.
Summary: 2 Safe | 0 Needs Review | 0 Blocked
✅ Safe
Brief or no lock, backwards compatible
web_analytics.0015_heatmap_page_history
└─ #1 ✅ CreateModel
Creating new table is safe
model: HeatmapScreenshotHistory
└─ #2 ✅ CreateModel
Creating new table is safe
model: HeatmapCaptureRequest
└─ #3 ✅ AddField
Adding nullable field requires brief lock
model: savedheatmap, field: last_viewed_at
└─ #4 ✅ AddField
Adding nullable field requires brief lock
model: savedheatmap, field: next_history_capture_at
└─ #5 ✅ AddField
Adding nullable field requires brief lock
model: savedheatmap, field: history_configuration_revision
└─ #6 ✅ AddField
Adding nullable field requires brief lock
model: heatmapscreenshothistory, field: latest_request
│
└──> ℹ️ INFO:
ℹ️ Skipped operations on newly created tables (empty tables
don't cause lock contention).
web_analytics.0016_savedheatmap_last_viewed_idx
└─ #1 ✅ SafeAddIndexConcurrently
PostHog concurrent-index helper: idempotent (timeout disabling + invalid-leftover recovery)
model: savedheatmap, index: heatmap_last_viewed_idx
📚 How to Deploy These Changes Safely
AddField:
This operation acquires a brief lock but doesn't rewrite the table.
Deployment uses lock timeouts with automatic retries, so lock contention will cause retries rather than connection pile-up.
Last updated: 2026-10-09 19:02 UTC (45a60a5)
|
Risk: No findings This delta keeps page-history images out of the exports system: it removes the full/thumbnail ExportedAsset FKs and the image/jpeg format addition, adding a has_thumbnail boolean and a revision-based has_content property instead. The models remain team-scoped, fail-closed, and unreachable by any request path, so the change only shrinks the surface; no new security issues. Sentinel reviewed |
a3f7af4 to
45a60a5
Compare
|
😎 Stack merged successfully - details. |
Problem
Page history keeps one screenshot per saved heatmap per day. It needs tables for those days and for the capture requests that fill them.
Part 3 of 9 in the heatmap page history stack. Stacked on #114214. Needs a human reviewer, because stamphog never approves migrations.
Changes
heatmap_screenshot_history/. A day points at its image throughrevision, the request that produced it.Underneath: both models join the IDOR semgrep rule.
Schema
posthog_savedheatmap(existing table, 3 new columns)last_viewed_atnext_history_capture_athistory_configuration_revisionposthog_heatmapscreenshothistory(new table)One row per saved heatmap per project-local day. The page history strip lists these rows.
idteam_idposthog_teamheatmap_idcaptured_onday_start,day_endtimezonewidthrevisionhas_thumbnailstatusok,pendingorfailedtriggerscheduled(daily run) ormanual(capture now)captured_atfailure_cause,page_statuslatest_request_idexpires_atcreated_at,updated_atposthog_heatmapcapturerequest(new table)One row per render attempt. A request copies its render inputs at enqueue time, so a later settings change cannot alter a render in flight.
idrevisionit publishesteam_idheatmap_idhistory_idstatequeued,running,succeeded,failedorcancelledtriggerscheduledormanualcaptured_on,timezone,day_start,day_endurl,width,block_consent_modalsinput_signatureclaim_iddeadlinefailure_cause,page_statuscreated_at,completed_atcompleted_atis nullableexpires_atConstraints and indexes
heatmap_history_one_per_dayUNIQUE (heatmap_id, captured_on)heatmap_history_one_scheduledUNIQUE (heatmap_id, captured_on) WHERE trigger = 'scheduled'heatmap_request_global_idx(trigger, created_at)heatmap_request_team_idx(team_id, captured_on, trigger)heatmap_request_live_idx(deadline) WHERE state IN ('queued', 'running')heatmap_last_viewed_idx(last_viewed_at)Migrations
0015_heatmap_page_historysavedheatmapcolumns0016_savedheatmap_last_viewed_idxheatmap_last_viewed_idxwithSafeAddIndexConcurrently(atomic = False)CREATE INDEX CONCURRENTLY, so writes toposthog_savedheatmapcontinue while it buildsHow did you test this code?
makemigrations --checkis clean. No new tests: the models have no behavior until the pipeline PR, which tests them.Release status
Docs update
None.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Agent: Claude Code, Claude Fable 5.1 and Claude Opus 5.5
Skills invoked: /django-migrations, /writing-pr-descriptions. CodeRabbit CLI ran with
--deepand found nothing. An earlier version stored images as exports assets, with a companion exports PR (#114216, closed). Page history now stays inside web analytics.