Repository navigation
feat(replay-vision): render scanner reasoning as markdown - #90347
Conversation
|
😎 Merged successfully - details. |
🦔 Hogbox preview · ✅ ready▶ Open the preview
commit |
🤖 CI report
|
| Function | Location | Complexity | Limit |
|---|---|---|---|
ReplayObservationSceneComponent |
products/replay_vision/frontend/observations/ReplayObservation.tsx:144 |
98 | 10 |
ObservationPrimaryOutput |
products/replay_vision/frontend/components/ObservationCard.tsx:114 |
53 | 10 |
ObservationDockCard |
products/replay_vision/frontend/components/ObservationCard.tsx:401 |
20 | 10 |
observationClipboardText |
products/replay_vision/frontend/utils/observation.ts:218 |
17 | 10 |
ReplayVisionScanWidget |
products/posthog_ai/frontend/components/tool/widgets/ReplayVisionScanWidget.tsx:32 |
12 | 10 |
ObservationStatusTag |
products/replay_vision/frontend/components/ObservationCard.tsx:27 |
12 | 10 |
⚠️ Comment density — 5% of added code lines are comments (27 of 510)
This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.
Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.
Files with the most added comment lines:
| File | Comment lines | Added lines |
|---|---|---|
frontend/src/lib/lemon-ui/LemonMarkdown/LemonMarkdown.tsx |
10 | 55 |
products/replay_vision/frontend/utils/citations.ts |
6 | 24 |
products/replay_vision/frontend/components/CitedMarkdown.stories.tsx |
4 | 70 |
products/replay_vision/frontend/components/CitedMarkdown.tsx |
2 | 29 |
products/replay_vision/frontend/utils/markdown.ts |
2 | 46 |
products/posthog_ai/frontend/components/tool/widgets/ReplayVisionScanWidget.tsx |
1 | 4 |
products/replay_vision/frontend/components/TimestampCitation.tsx |
1 | 22 |
products/replay_vision/frontend/utils/observation.ts |
1 | 6 |
This check does not block merging. It updates on every push and clears when the share drops.
⚠️ Bundle size — 🔺 +492 B (+0.0%)
Uncompressed size of every built .js bundle, compared against the base branch.
Total: 68.65 MiB · 🔺 +492 B (+0.0%)
No file changed by more than 1000 B.
Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report
✅ Eager graph — within budget
How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.
| Root | Eager (shipped) | Δ vs base | Budget |
|---|---|---|---|
entry (logged-out pages, app bootstrap)src/index.tsx |
1.50 MiB · 22 files | no change | ███░░░░░░░ 33.2% of 4.51 MiB |
logged-out boot: index + App + bootApp (preloaded by every page, including /login)src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts |
5.24 MiB · 668 files | no change | █████████░ 93.1% of 5.63 MiB |
authenticated shell (every logged-in page)src/scenes/AuthenticatedShell.tsx |
7.66 MiB · 2,677 files | 🔺 +550 B (+0.0%) | █████████░ 87.3% of 8.77 MiB |
🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
Largest files eagerly shipped from src/index.tsx
| Size | File |
|---|---|
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 24.6 KiB | ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js |
| 6.3 KiB | ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js |
| 4.5 KiB | ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js |
| 3.9 KiB | ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js |
| 1.4 KiB | ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js |
| 1.3 KiB | src/RootErrorBoundary.tsx |
| 1.0 KiB | src/index.tsx |
| 912 B | ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js |
| 854 B | src/scenes/ChunkLoadErrorBoundary.tsx |
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
| Size | File |
|---|---|
| 281.8 KiB | ../node_modules/.pnpm/posthog-js@1.428.6_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 255.4 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 104.2 KiB | src/lib/api.ts |
| 79.0 KiB | src/products.tsx |
| 68.8 KiB | src/lib/lemon-ui/icons/icons.tsx |
| 62.5 KiB | src/lib/utils/eventUsageLogic.ts |
| 38.8 KiB | ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js |
| 33.9 KiB | ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js |
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
| Size | File |
|---|---|
| 281.8 KiB | ../node_modules/.pnpm/posthog-js@1.428.6_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs |
| 267.7 KiB | ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js |
| 255.4 KiB | src/taxonomy/core-filter-definitions-by-group.json |
| 153.8 KiB | ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js |
| 126.8 KiB | ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js |
| 104.2 KiB | src/lib/api.ts |
| 95.8 KiB | ../packages/quill/packages/quill/dist/index.js |
| 93.3 KiB | ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js |
| 90.6 KiB | ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js |
| 79.0 KiB | src/products.tsx |
Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479
✅ Toolbar bundle — eager 2.29 MiB within budget
What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.
| Metric | Size | Δ vs base | Budget |
|---|---|---|---|
| Eager (shipped) entry + static imports |
2.29 MiB · 18 files | no change | ████░░░░░░ 40.0% of 5.72 MiB |
| Deferred (lazy) | 2.11 MiB · 45 files | no change | n/a — loads on demand |
Loader dist/toolbar.js |
1.1 KiB | no change | █░░░░░░░░░ 5.8% of 19.5 KiB |
Largest eagerly-shipped chunks
| Size | File |
|---|---|
| 756.4 KiB | dist/toolbar/toolbar-app-LA4JX7FS.css |
| 608.8 KiB | dist/toolbar/chunk-chunk-P3PF5T4P.js |
| 484.7 KiB | dist/toolbar/chunk-chunk-RPV3NYRJ.js |
| 136.6 KiB | dist/toolbar/chunk-chunk-6MQWLXON.js |
| 131.8 KiB | dist/toolbar/chunk-chunk-FDH2IBXT.js |
| 72.2 KiB | dist/toolbar/toolbar-app-VLT4VAGU.js |
| 69.0 KiB | dist/toolbar/chunk-chunk-TSAL54PB.js |
| 35.6 KiB | dist/toolbar/chunk-chunk-DTB5C7AH.js |
| 21.0 KiB | dist/toolbar/chunk-chunk-35CVENAR.js |
| 6.8 KiB | dist/toolbar/chunk-chunk-DV7IWQNF.js |
Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile
✅ Dist folder size — 🔺 +26.9 KiB (+0.0%)
Total size of the built frontend/dist folder (all assets), compared against the base branch.
Total: 1458.18 MiB · 🔺 +26.9 KiB (+0.0%)
⚠️ Playwright — 1 flaky
🎭 Playwright report · View test results →
- Add a funnel insight, configure steps, and verify conversion rates (chromium)
These issues are not necessarily caused by your changes.
Annoyed by this section? Help fix flakies and failures and it will go green!
✅ Backend coverage — all changed backend lines covered
🧪 Backend test coverage
Patch coverage — changed backend lines (products + core): ████████████████████ 100.0% (62 / 62)
All changed backend lines are covered ✅
Per-product line coverage (touched products)
| Product | Coverage | Lines |
|---|---|---|
platform_features |
██░░░░░░░░░░░░░░░░░░ 12.1% |
7 / 58 |
warehouse_sources_queue |
██████░░░░░░░░░░░░░░ 29.1% |
92 / 316 |
data_tools |
████████████░░░░░░░░ 62.1% |
90 / 145 |
demo |
████████████░░░░░░░░ 62.1% |
1,660 / 2,671 |
ai_gateway |
███████████████░░░░░ 75.0% |
9 / 12 |
batch_exports |
████████████████░░░░ 80.1% |
19,507 / 24,368 |
apm |
█████████████████░░░ 83.3% |
1,234 / 1,481 |
wizard |
█████████████████░░░ 85.3% |
2,779 / 3,259 |
cdp |
█████████████████░░░ 85.4% |
4,349 / 5,094 |
signals |
█████████████████░░░ 85.5% |
36,471 / 42,645 |
product_tours |
██████████████████░░ 88.0% |
1,312 / 1,491 |
notebooks |
██████████████████░░ 88.5% |
11,879 / 13,429 |
data_warehouse |
██████████████████░░ 88.9% |
12,824 / 14,424 |
business_knowledge |
██████████████████░░ 89.2% |
4,469 / 5,008 |
cohorts |
██████████████████░░ 89.3% |
7,543 / 8,450 |
dashboards |
██████████████████░░ 89.6% |
6,837 / 7,631 |
engineering_analytics |
██████████████████░░ 90.1% |
8,391 / 9,310 |
streamlit_apps |
██████████████████░░ 90.2% |
2,517 / 2,790 |
tasks |
██████████████████░░ 90.3% |
65,989 / 73,098 |
data_modeling |
██████████████████░░ 90.4% |
9,350 / 10,347 |
managed_warehouse |
██████████████████░░ 90.7% |
9,417 / 10,378 |
exports |
██████████████████░░ 90.8% |
9,123 / 10,050 |
alerts |
██████████████████░░ 90.9% |
5,242 / 5,765 |
links |
██████████████████░░ 91.0% |
192 / 211 |
mcp_analytics |
██████████████████░░ 91.2% |
4,411 / 4,837 |
canvas |
██████████████████░░ 91.3% |
5,033 / 5,514 |
conversations |
██████████████████░░ 91.3% |
21,739 / 23,816 |
visual_review |
██████████████████░░ 91.7% |
7,567 / 8,250 |
error_tracking |
██████████████████░░ 92.0% |
13,495 / 14,669 |
notifications |
██████████████████░░ 92.5% |
1,120 / 1,211 |
early_access_features |
███████████████████░ 92.6% |
1,331 / 1,438 |
managed_migrations |
███████████████████░ 92.7% |
1,576 / 1,700 |
slack_app |
███████████████████░ 92.7% |
12,684 / 13,676 |
surveys |
███████████████████░ 93.1% |
6,180 / 6,640 |
mcp_registry |
███████████████████░ 93.2% |
1,471 / 1,578 |
web_analytics |
███████████████████░ 93.3% |
19,324 / 20,706 |
stamphog |
███████████████████░ 93.4% |
6,543 / 7,009 |
context_layer |
███████████████████░ 93.4% |
2,616 / 2,801 |
mcp_store |
███████████████████░ 93.5% |
8,063 / 8,621 |
reminders |
███████████████████░ 93.5% |
477 / 510 |
marketing_analytics |
███████████████████░ 93.5% |
17,452 / 18,657 |
posthog_ai |
███████████████████░ 93.6% |
1,395 / 1,491 |
ai_observability |
███████████████████░ 93.8% |
18,255 / 19,463 |
approvals |
███████████████████░ 93.9% |
3,570 / 3,803 |
billing_alerts |
███████████████████░ 94.1% |
2,079 / 2,210 |
endpoints |
███████████████████░ 94.2% |
9,072 / 9,627 |
review_hog |
███████████████████░ 94.3% |
10,214 / 10,826 |
tracing |
███████████████████░ 94.5% |
2,726 / 2,885 |
legal_documents |
███████████████████░ 94.7% |
2,082 / 2,199 |
workflows |
███████████████████░ 94.7% |
11,801 / 12,455 |
autoresearch |
███████████████████░ 95.0% |
934 / 983 |
customer_analytics |
███████████████████░ 95.1% |
19,308 / 20,303 |
annotations |
███████████████████░ 95.3% |
804 / 844 |
product_analytics |
███████████████████░ 95.3% |
19,328 / 20,280 |
actions |
███████████████████░ 95.4% |
749 / 785 |
replay_vision |
███████████████████░ 95.5% |
22,800 / 23,868 |
skills |
███████████████████░ 95.6% |
5,536 / 5,791 |
experiments |
███████████████████░ 95.6% |
30,259 / 31,643 |
access_control |
███████████████████░ 95.7% |
6,247 / 6,527 |
feature_flags |
███████████████████░ 95.7% |
20,313 / 21,220 |
messaging |
███████████████████░ 95.9% |
3,680 / 3,839 |
growth |
███████████████████░ 96.0% |
7,728 / 8,053 |
logs |
███████████████████░ 96.0% |
13,950 / 14,532 |
revenue_analytics |
███████████████████░ 96.3% |
1,841 / 1,911 |
user_interviews |
███████████████████░ 96.5% |
2,651 / 2,747 |
warehouse_sources |
███████████████████░ 97.2% |
409,791 / 421,671 |
data_quality |
████████████████████ 97.6% |
4,229 / 4,335 |
analytics_platform |
████████████████████ 98.2% |
2,381 / 2,425 |
metrics |
████████████████████ 98.2% |
3,576 / 3,640 |
data_catalog |
████████████████████ 98.3% |
3,267 / 3,322 |
pulse |
████████████████████ 98.4% |
2,028 / 2,060 |
live_debugger |
████████████████████ 99.2% |
621 / 626 |
field_notes |
████████████████████ 99.4% |
167 / 168 |
Report-only. Patch coverage = changed backend lines covered vs origin/master. Sorted lowest first.
Known gaps: lines covered only by Temporal tests show as uncovered; core line numbers may drift if master changed the same file.
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
|
Two independent reviewers (parameterai and veria-ai bots) both identify the same concrete, verified gap: defangMarkdownLinks/flatten_markdown only strip inline
Gate mechanics and policy version
|
|
✅ Visual changes approved by hayne — baseline updated in 6 new. |
Scanner reasoning that carries a bold lead-in or a bullet list now renders as structure on the observation card, instead of showing the raw syntax. The same string feeds surfaces that read it as one line of prose: Slack alert messages, the Max and Signals search snippets, the group-summary feed, the search embeddings, the seekbar marks. Each gets a flattened copy, so a markdown bullet cannot arrive there as a literal row marker. Generated-By: PostHog Desktop Task-Id: 072e2d37-89a7-4ab7-90e1-a71f90862523
…egex Stripping link syntax from the markdown source only covered inline `[x](url)`. Reference links, `[x][ref]` with the definition elsewhere in the document, went through untouched and rendered as a live external anchor, as did an image by reference via the untrusted-image click-to-open fallback. LemonMarkdown gains `disableLinks`, which renders every link as its label instead. No markdown form can get past it, because it simply never emits an anchor. The source-level defanging goes away with it. The flatteners keep their own reference-link handling — there the syntax is a readability problem in a one-line snippet, not a live link. Generated-By: PostHog Desktop Task-Id: 072e2d37-89a7-4ab7-90e1-a71f90862523
ba85d48 to
9dfc9de
Compare
6 updated Run: 2713d41a-b311-4426-b3f5-cb1a4be02dfb
|
/trunk merge |
Problem
A person reading a Replay Vision observation sees
**Checkout blocked at payment**with the asterisks, and a-in front of every bullet. Scanners already write a lead-in and a list when a finding has parts, and the Model reasoning card renders all of it as one flat paragraph, so the structure blends in.Rendering markdown is the easy half. The same
reasoningstring feeds six surfaces that read it as one line of prose, and each collapses whitespace without stripping syntax. A markdown bullet arrives in a Slack alert as a literal-at the start of a line it does not own, and**tokens go into the vectors that semantic search ranks on.Changes
**and-**and-**and-line-clampcounts lines inside one block, so it cannot trim a rendered heading and list to two.LemonMarkdowngainsdisableLinks, which renders every link as its label;CitedMarkdownsets it. Enforced at the renderer rather than by stripping syntax, because markdown reaches a link through inline[x](url), reference[x][ref], a bare URL autolinked by remark-gfm, or an untrusted image falling back to a link — a sanitizer upstream of the parser has to out-guess all of them, while this never emits an anchor.LemonMarkdowngainsrenderTimestampRef, at:<ms>link target that mirrors the existingchart:reference. A remark plugin was the alternative; the link target reuses a mechanism already in the component and degrades to a readable timestamp where no player is on screen.CitedTextintoTimestampCitation, shared by both renderers.Note
The scanner prompts are unchanged and still ask for one paragraph. The model already writes structure sometimes, which is what this PR renders. Loosening the prompt is a follow-up, so it can be measured against a UI that already shows the difference.
Before and after, from the new Storybook story:
How did you test this code?
react-markdownis mocked in Jest because it is ESM-only, so no Jest test can prove the rendering works. The stories were rendered in headless Chromium against a local Storybook instead: bold became<strong>, bullets became<ul><li>, both citations became seek buttons, and the page reported no anchors and no errors.The no-clickable-link property is guarded by the
HostileLinksstory, whoseplayasserts zero anchors and runs in a real browser undertest-storybookin CI. Both halves of that guard were checked rather than assumed: inverting the assertion made it fail fromplay, and removingdisableLinksbrought back 5 live anchors — the inline link,[click here][ref],![a banner][img]via the untrusted-image link fallback, the autolink, and the bare URL. So the fixture reproduces the exposure and the assertion catches it.New tests and the regression each catches:
plain_snippetcases intest_observation_formatting.py: a bullet list reaching a Slack alert or the synthesis feed as a literal-row, andsnake_caseor3 * 4being eaten as emphasis. Extended the existing forgery test rather than adding a second one.flatten_markdownkeeping paragraph breaks: embeddings take the whole flattened string, so a strip that also folded the lines would change what is indexed.citedMarkdowncases incitations.test.ts: a bullet or bold run that spans a citation being cut into two markdown documents, a citation after a!parsing as an image, and the prose being altered on its way to the renderer now that sanitizing is the renderer's job.[x][ref]usage reaching a Slack alert with its brackets intact, and the inverse — a sentence like[Save]: clicked twice before it tookbeing mistaken for a link reference definition and dropped.Not run: the full backend suite. The affected files were covered locally against Postgres and ClickHouse.
Automatic notifications
Docs update
None. No documented workflow changes.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written by the PostHog Slack app (Claude Opus) from a request in a Slack thread asking whether the Model reasoning field could use markdown, and to cover the full stack without cluttering the Scouts or Alerts flow.
Skills invoked:
/writing-tests,/writing-ui-components,/writing-pr-descriptions.Two decisions worth flagging. A bespoke renderer inside the product was the first design; the
react/forbid-elementslint rule reservesReactMarkdowntoLemonMarkdown, which settled it in favour of extending the shared component. Link handling started as regex defanging of the source; review (1, 2) showed reference links went straight through it, so it moved to the render boundary and the regexes came out.All sample text in the tests, the story and the screenshots is invented. No customer material was used.
Created with PostHog from a Slack thread