fix(replay-vision): trim repeated fields from scanner observation rows by posthog[bot] · Pull Request #95794 · PostHog/posthog

posthog · 2026-09-06T15:57:48Z

Problem

An agent asking "what has this scanner found?" cannot call vision-scanners-observations-list at its documented default. The page overflows the exec token cap, so every caller has to guess a small limit first.
Three fields on each row carry no per-row information:

Field	Why it is waste
`scanner_snapshot.scanner_config.prompt`	Byte-identical on every row of a scanner's page, and the largest field on it
`scanner_snapshot.query`	Byte-identical on every row too
`scanner_result.model_output.reasoning_segments`	Restates the flat `reasoning`, interleaved with timestamp chips

Verified on a live monitor scanner on 2026-09-06: a 3-row page repeated the same 3,071-character prompt three times, over 9,200 characters of repetition from one static field.
Reported on 2026-07-04. #68360 implemented a fix and the stale bot closed it unmerged on 2026-08-13, with no review objection. Nothing has changed for a caller since.

Changes

The tool's default page is now 20 observations, and its rows no longer repeat the prompt, the query, or the duplicated reasoning segments. An agent can call it with no arguments and read the result.
The tool description and agent note now send a caller to vision-scanners-observations-get for one observation's full prompt and citation timestamps, so deep-linking a moment still works, at one extra call per finding.
The trim sits in the MCP layer rather than the serializer, which is where #68360 put it. That closed PR would have broken four app surfaces that read these same fields off list rows: the scanner observations table and calibration tab (prompt tooltip on monitor and scorer rows), the replay dock and player sidebar (CitedText reasoning and seekbar citation marks), and the "copy all observations" clipboard export. The REST API keeps serving every field.
Mechanical: regenerated services/mcp/schema/*.json, src/tools/generated/replay_vision.ts, and the tool-schema snapshot from tools.yaml.

+        param_overrides:
+            limit:
+                default: 20
+        response:
+            exclude:
+                - scanner_snapshot.scanner_config.prompt
+                - scanner_snapshot.query
+                - scanner_result.model_output.reasoning_segments

How did you test this code?

New services/mcp/tests/unit/vision-scanner-observations-list-trimming.test.ts drives the generated handler over a mocked API response. It catches a regression no existing test covers: the exclude paths silently no-op if a path stops matching the response shape, which is exactly how this defect would return. The schema-snapshot test only sees the input schema, and tests/unit/utils.test.ts only covers the omitResponseFields helper in isolation.
Confirmed the test fails without the fix: removing the generated filter turns the first case red, so it is not asserting its own configuration.
Ran tests/unit/vision-scanner-observations-list-trimming.test.ts, tool-schema-snapshots, generate-tools, and utils in services/mcp, plus lint-tool-names and hogli ci:preflight --fix (no failures).
Not checked: pnpm typecheck in services/mcp cannot run clean in this sandbox because the nested @posthog/quill workspace is unbuilt, which errors on every product's MCP app. No error referenced the two files changed here.
Reproduced the symptom against a live scanner through the MCP tool, but did not re-run the tool against a rebuilt server, since the change is in the checked-in generated handler.

Automatic notifications

Publish to changelog?

Docs update

None. The tool description and agent note are the user-facing surface, and both are updated here.

🤖 Agent context

Autonomy: Fully autonomous

Written by Claude Code from a PostHog inbox report. Skills invoked: /writing-simplified-technical-english, /writing-pr-descriptions. Repo conventions for MCP tools came from services/mcp/CONTRIBUTING.md and docs/published/handbook/engineering/ai/implementing-mcp-tools.md.

Two approaches were tried before this one. The first reapplied #68360 in the DRF serializer, gated on self.action == "list". Tracing the frontend consumers showed that trims real features off four app surfaces, so it was reverted. The response.exclude mechanism in tools.yaml reaches only agent callers and is what the earlier list-overflow fixes used, so the app is untouched by design.

Two details in the report needed correcting on the way. It named vision-scanners-observations-retrieve; the tool is vision-scanners-observations-get. It also did not account for the tool's agent note, added after #68360 was written, which tells agents to read reasoning_segments off list rows to deep-link a finding. Dropping that field without rewriting the note would have left the tool documenting a field it no longer returns, so the note now describes the retrieve hop.

No duplicate: gh pr list --state open --search over both the tool name and "replay vision observations list" found no open PR fixing this. #89485 is adjacent (frontend reasoning hover) and does not touch the MCP layer.

Public artifact: the report and the live reproduction both involved real customer session content. Nothing from either reached this PR. The test fixture was written from a list of the properties it has to exercise, with the real payload closed, using an invented scanner and invented prose.

Created with PostHog Desktop from this inbox report.

The scanner-scoped observations list re-embedded the scanner's prompt and recordings query on every row, and restated each row's reasoning a second time as reasoning_segments. On the default page this overflowed an agent's token budget, so the tool could not be called without first guessing a small limit. Declare the three paths under `response.exclude` on the MCP tool, and default its `limit` to 20. The trim sits in the MCP layer because the app's own observations table, calibration tab, replay dock, and clipboard export all read those fields from list rows; the REST API keeps serving them. Point the tool description and agent note at vision-scanners-observations-get for one observation's full prompt and citation timestamps. Generated-By: PostHog Desktop Task-Id: cfd06794-5dc1-4cdb-b3d1-f66d05273694

trunk-io · 2026-09-06T15:57:51Z

Merging to master in this repository is managed by Trunk.

To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

posthog · 2026-09-06T15:57:52Z

🦔 PostHog Review reviewed this pull request

Found 0 must fix, 4 should fix, 1 consider.

Published 5 findings (view the review).

Resolved comments: 4 fixed, 1 declined

github-actions · 2026-09-06T15:59:53Z

🤖 CI report

⚠️

Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Bundle size — no change

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 68.62 MiB · no change

No file changed by more than 1000 B.

_{Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report}

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root	Eager (shipped)	Δ vs base	Budget
entry (logged-out pages, app bootstrap) `src/index.tsx`	1.50 MiB · 22 files	no change	`███░░░░░░░` 33.2% of 4.51 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login) `src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts`	5.25 MiB · 668 files	no change	`█████████░` 93.3% of 5.63 MiB
authenticated shell (every logged-in page) `src/scenes/AuthenticatedShell.tsx`	7.67 MiB · 2,677 files	no change	`█████████░` 87.4% of 8.77 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx

Size	File
126.8 KiB	`../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js`
24.6 KiB	`../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js`
6.3 KiB	`../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js`
4.5 KiB	`../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js`
3.9 KiB	`../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js`
1.4 KiB	`../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js`
1.3 KiB	`src/RootErrorBoundary.tsx`
1000 B	`src/index.tsx`
912 B	`../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js`
854 B	`src/scenes/ChunkLoadErrorBoundary.tsx`

Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts

Size	File
276.1 KiB	`../node_modules/.pnpm/posthog-js@1.427.2_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs`
267.7 KiB	`../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js`
255.2 KiB	`src/taxonomy/core-filter-definitions-by-group.json`
126.8 KiB	`../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js`
104.4 KiB	`src/lib/api.ts`
79.1 KiB	`src/products.tsx`
68.8 KiB	`src/lib/lemon-ui/icons/icons.tsx`
62.5 KiB	`src/lib/utils/eventUsageLogic.ts`
38.8 KiB	`../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js`
33.9 KiB	`../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js`

Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx

Size	File
276.1 KiB	`../node_modules/.pnpm/posthog-js@1.427.2_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs`
267.7 KiB	`../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js`
255.2 KiB	`src/taxonomy/core-filter-definitions-by-group.json`
153.8 KiB	`../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js`
126.8 KiB	`../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js`
104.4 KiB	`src/lib/api.ts`
95.8 KiB	`../packages/quill/packages/quill/dist/index.js`
93.3 KiB	`../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js`
90.6 KiB	`../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js`
79.1 KiB	`src/products.tsx`

_{Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479}

✅ Toolbar bundle — eager 2.28 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric	Size	Δ vs base	Budget
Eager (shipped) entry + static imports	2.28 MiB · 18 files	no change	`████░░░░░░` 39.9% of 5.72 MiB
Deferred (lazy)	2.11 MiB · 45 files	no change	n/a — loads on demand
Loader `dist/toolbar.js`	1.1 KiB	no change	`█░░░░░░░░░` 5.8% of 19.5 KiB

Largest eagerly-shipped chunks

Size	File
756.3 KiB	`dist/toolbar/toolbar-app-4464UOOM.css`
603.1 KiB	`dist/toolbar/chunk-chunk-NCE4QL3M.js`
484.7 KiB	`dist/toolbar/chunk-chunk-D33R7XDZ.js`
136.5 KiB	`dist/toolbar/chunk-chunk-BXI27EUT.js`
131.8 KiB	`dist/toolbar/chunk-chunk-FDH2IBXT.js`
72.2 KiB	`dist/toolbar/toolbar-app-CJMMARKZ.js`
69.0 KiB	`dist/toolbar/chunk-chunk-TSAL54PB.js`
35.6 KiB	`dist/toolbar/chunk-chunk-VLYPUESV.js`
21.0 KiB	`dist/toolbar/chunk-chunk-LPBZV4EM.js`
6.8 KiB	`dist/toolbar/chunk-chunk-DV7IWQNF.js`

_{Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile}

✅ Dist folder size — no change

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1457.43 MiB · no change

ℹ️ MCP UI apps size — 33 app(s), 17665.2 KB JS

Built size of each MCP UI app (main.js + styles.css).

App	JS	CSS
debug	600.0 KB	195.2 KB
action	458.2 KB	195.2 KB
action-list	564.9 KB	195.2 KB
cohort	457.2 KB	195.2 KB
cohort-list	563.8 KB	195.2 KB
email-template	457.0 KB	195.2 KB
error-details	472.9 KB	195.2 KB
error-issue	457.9 KB	195.2 KB
error-issue-list	564.7 KB	195.2 KB
experiment	562.0 KB	195.2 KB
experiment-list	565.6 KB	195.2 KB
experiment-results	567.1 KB	195.2 KB
feature-flag	567.6 KB	195.2 KB
feature-flag-list	571.4 KB	195.2 KB
feature-flag-testing	461.4 KB	195.2 KB
inline-scan	457.7 KB	195.2 KB
insight-actors	563.0 KB	195.2 KB
invite-email-preview	456.4 KB	195.2 KB
llm-costs	560.0 KB	195.2 KB
session-recording	459.0 KB	195.2 KB
survey	458.8 KB	195.2 KB
survey-global-stats	562.7 KB	195.2 KB
survey-list	565.5 KB	195.2 KB
survey-stats	562.7 KB	195.2 KB
trace-span	457.6 KB	195.2 KB
trace-span-list	564.7 KB	195.2 KB
vision-observation-list	563.9 KB	195.2 KB
workflow	457.5 KB	195.2 KB
workflow-list	564.2 KB	195.2 KB
loops-review	461.9 KB	195.2 KB
query-results	756.3 KB	195.2 KB
render-ui	839.5 KB	195.2 KB
visual-review-snapshots	462.0 KB	195.2 KB

stamphog

The review did not complete.

Stamphog hit an error and produced no verdict for this commit.

Push a new commit to try again.

posthog · 2026-09-06T16:19:35Z

PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

posthog

PostHog Review

Found 4 should fix, 1 consider.

The observations-list agent note told every caller to read citation timestamps from `reasoning_segments`. Summarizer output has no such field; its citation segments are `summary_segments`. The note now names the field for each scanner type. It also claimed the list carries no citation timestamps. That is false for summarizer rows, whose `summary_segments` the new exclude path does not match, so the claim is dropped. Regenerated `services/mcp/src/tools/generated/replay_vision.ts` from `tools.yaml`. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47

…sion The observations-list description said the snapshot prompt and query are the same on every row. They are frozen per observation, and editing either bumps `scanner_version`, so a page written across an edit holds both values. An agent reading that copy attributes older findings to the current prompt. The description now scopes the repetition to one scanner version, says an edit bumps `scanner_version`, and tells a caller spanning versions to read one observation per version before comparing findings. Regenerated the MCP tool schema JSON from `tools.yaml`. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47

Citation segments are written as `{citation_field}_segments`, and a summarizer cites `summary`, so its duplicate lives in `summary_segments`. The exclude path matched only `reasoning_segments`, so a summarizer page still shipped each body twice. That is the scanner type with the largest free text, so the page-size reduction this change targets missed it. Adds the `summary_segments` exclude path, and a summarizer fixture that proves the row keeps `summary` and loses `summary_segments`. The existing fixtures are all monitor rows, so nothing covered the second field name. The test fails without the exclude path. The description now names the dropped segment field per scanner type. Regenerated the MCP handler and tool schema JSON from `tools.yaml`. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47

…ieve The bundled `exploring-replay-vision-observations` skill names `vision-scanners-observations-list` as the workhorse, then reads citation segments straight off those rows. Trimming the segments off that page leaves the skill prescribing a field the tool no longer returns, so an agent following it drops the deep links the section calls the point of the step. The section now says the scanner-scoped list leaves the segments off, and to read the observation with `vision-scanners-observations-get` before citing it. It also names the segment field per scanner type, and records that the session-scoped tools still return segments, so that path needs no extra call. The skill is seeded per team as an external companion skill, so scout runs load this text. Generated-By: PostHog Desktop Task-Id: 3458d9f4-4d0e-4a22-a7d3-1386f65d3e47

scheduled-actions-posthog · 2026-09-14T07:44:52Z

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

scheduled-actions-posthog · 2026-09-21T07:52:51Z

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.

posthog Bot added skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs and removed skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs labels Sep 6, 2026

chore: update OpenAPI generated types

db591a8

stamphog Bot reviewed Sep 6, 2026

View reviewed changes

posthog Bot commented Sep 6, 2026

View reviewed changes

posthog Bot and others added 5 commits September 6, 2026 16:25

chore: update OpenAPI generated types

b71ecca

posthog Bot mentioned this pull request Sep 8, 2026

perf(replay-vision): bound the scanner list response to metadata #96281

Closed

1 task

arnohillen mentioned this pull request Sep 8, 2026

feat(replay-vision): verify monitor yes verdicts with extra draws #96742

Merged

1 task

scheduled-actions-posthog Bot added the stale label Sep 14, 2026

This was referenced Sep 19, 2026

fix(replay-vision): give observation lists a compact text projection #103321

Merged

fix(mcp): narrow projected observation rows in both response channels #103437

Closed

scheduled-actions-posthog Bot closed this Sep 21, 2026

Navigation Menu

Repository navigation

fix(replay-vision): trim repeated fields from scanner observation rows - #95794

fix(replay-vision): trim repeated fields from scanner observation rows#95794
posthog[bot] wants to merge 7 commits into
masterfrom
posthog-self-driving/vision-scanners-observations-list-still-57b621

posthog Bot commented Sep 6, 2026

Uh oh!

trunk-io Bot commented Sep 6, 2026

Uh oh!

posthog Bot commented Sep 6, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 6, 2026 •

edited

Loading

Uh oh!

stamphog Bot left a comment

Uh oh!

posthog Bot commented Sep 6, 2026

Uh oh!

posthog Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

scheduled-actions-posthog Bot commented Sep 14, 2026

Uh oh!

scheduled-actions-posthog Bot commented Sep 21, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

Conversation

posthog Bot commented Sep 6, 2026

Problem

Changes

How did you test this code?

Automatic notifications

Docs update

🤖 Agent context

Uh oh!

trunk-io Bot commented Sep 6, 2026

Uh oh!

posthog Bot commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦔 PostHog Review reviewed this pull request

Uh oh!

github-actions Bot commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 CI report

Uh oh!

stamphog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

posthog Bot commented Sep 6, 2026

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

PostHog Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

scheduled-actions-posthog Bot commented Sep 14, 2026

Uh oh!

scheduled-actions-posthog Bot commented Sep 21, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

posthog Bot commented Sep 6, 2026 •

edited

Loading

github-actions Bot commented Sep 6, 2026 •

edited

Loading