Skip to content

chore(experiments): remove dead paths from the AI summary tool - #87799

Merged
trunk-io[bot] merged 4 commits into
masterfrom
experiments/max-summary-remove-dead-paths
Aug 24, 2026
Merged

trunk-io[bot] merged 4 commits into
masterfrom
experiments/max-summary-remove-dead-paths

Conversation

@jurajmajerik

Copy link
Copy Markdown
Contributor

Problem

The AI summary tool has two paths: fetch data itself, or format metric results supplied in frontend context. Nothing sends metric results anymore - the frontend deliberately sends only experiment_id - so _format_from_context is dead in production. check_data_freshness and the frontend_last_refresh context field are defined and tested but never called by the tool at all.

Changes

  • Remove _format_from_context; the tool always fetches via ExperimentSummaryDataService.
  • Remove check_data_freshness and FRESHNESS_THRESHOLD_SECONDS from the data service.
  • Stop sending frontend_last_refresh from SummarizeExperimentButton.
  • Rework the summary tool tests to mock the data service instead of passing metric results through context; drop the 5 freshness tests.

How did you test this code?

test_max_tools.py + test_experiment_summary_tool.py (51 passed). No new tests: this removes behavior; the existing assertions now run through the surviving fetch path, which keeps formatter coverage intact. Frontend typecheck left to CI (light worktree).

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code follow-up to #84316/#84318/#84320. Confirmed the context path is unreachable by checking that no frontend code sends primary_metrics_results and that the tool never calls check_data_freshness. Skills invoked: /wt, /writing-tests.

@jurajmajerik jurajmajerik self-assigned this Aug 24, 2026
@trunk-io

trunk-io Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@github-actions

github-actions Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

⚠️ Bundle size — 🔺 +1.5 KiB (+0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 68.74 MiB · 🔺 +1.5 KiB (+0.0%)

File Size Δ vs base
render-query/src/render-query/render-query.js 21.91 MiB 🔺 +1.7 KiB (+0.0%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.36 MiB · 22 files 🔺 +43 B (+0.0%) ███░░░░░░░ 30.2% of 4.51 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
8.75 MiB · 3,232 files 🟢 -10 B (-0.0%) █████████░ 90.1% of 9.71 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 [object Object] stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/ActivityLog/describers stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx
🟢 [object Object] stays out of src/scenes/AuthenticatedShell.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.5 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
3.9 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.4 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.3 KiB src/RootErrorBoundary.tsx
912 B ../node_modules/.pnpm/ieee754@1.2.1/node_modules/ieee754/index.js
789 B src/scenes/ChunkLoadErrorBoundary.tsx
762 B src/index.tsx
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
304.7 KiB ../node_modules/.pnpm/posthog-js@1.418.5/node_modules/posthog-js/dist/rrweb.js
267.7 KiB ../node_modules/.pnpm/@posthog+icons@0.38.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
250.7 KiB ../node_modules/.pnpm/posthog-js@1.418.5/node_modules/posthog-js/dist/module.js
246.9 KiB src/taxonomy/core-filter-definitions-by-group.json
154.2 KiB ../node_modules/.pnpm/re2js@0.4.1/node_modules/re2js/build/index.esm.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.4 KiB src/lib/api.ts
95.1 KiB ../packages/quill/packages/quill/dist/index.js
93.3 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Toolbar bundle — eager 2.24 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.24 MiB · 17 files 🔺 +43 B (+0.0%) ████░░░░░░ 39.1% of 5.72 MiB
Deferred (lazy) 2.10 MiB · 33 files no change n/a — loads on demand
Loader dist/toolbar.js 1.1 KiB no change █░░░░░░░░░ 5.8% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
743.5 KiB dist/toolbar/toolbar-app-C3UK5TL3.css
574.9 KiB dist/toolbar/chunk-chunk-6JQ6W2QX.js
484.6 KiB dist/toolbar/chunk-chunk-F4LOU7PS.js
133.8 KiB dist/toolbar/chunk-chunk-6IEOBUNT.js
131.8 KiB dist/toolbar/chunk-chunk-T5KY5WYR.js
71.2 KiB dist/toolbar/toolbar-app-J7XPYELN.js
69.0 KiB dist/toolbar/chunk-chunk-27JL52RE.js
35.6 KiB dist/toolbar/chunk-chunk-K2VT5S2P.js
20.9 KiB dist/toolbar/chunk-chunk-U4AORDOH.js
12.2 KiB dist/toolbar/chunk-chunk-PIK3PADE.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +125.1 KiB (+0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 1463.95 MiB · 🔺 +125.1 KiB (+0.0%)

✅ Playwright — all passed

All tests passed.

@github-actions

github-actions Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

🦔 Hogbox preview · ✅ ready

▶ Open the preview

🔑 Login test@posthog.com / 12345678 (demo data)
🧩 Running this PR's backend and frontend, on the PostHog :master base
🔗 Link stable across rebuilds — a re-push swaps the box underneath, the URL stays
🔒 Access tailnet only (PostHog VPN)
🛠️ Admin inspect & debug state in hogland
💤 Idle sleeps after ~30 min idle (snapshot to S3, zero node cost) and wakes on your next visit in ~30s, behind a brief "waking up" screen

commit 5acdbbf · box box-37b078b92070 · ready in 590s (push → usable) · build log · rebuilds on every push, torn down on close

@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team August 24, 2026 12:45
@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "chore(experiments): remove dead paths fr..." | Re-trigger Greptile

@trunk-io

trunk-io Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

@jurajmajerik jurajmajerik added the stamphog Request AI approval (no full review) label Aug 24, 2026
stamphog[bot]
stamphog Bot previously approved these changes Aug 24, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contained dead-code removal in an internal experiments AI tool, backed by updated and passing tests, by an author on the owning team — no risky-territory surface touched.

  • Author wrote 4% of the modified lines and has 65 merged PRs in these paths (familiarity MODERATE).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 99L, 3F substantive, 226L/5F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1c-medium (226L, 5F, two-areas, chore)
stamphog 2.0.0b4 .stamphog/policy.yml @ 3d616dd · reviewed head 3d616dd

@posthog

posthog Bot commented Aug 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ Visual changes approved by @jurajmajerik — baseline updated in 5acdbbf.

View this run in PostHog

1 changed.

…mary-remove-dead-paths

# Conflicts:
#	products/experiments/backend/max_tools.py
@stamphog
stamphog Bot dismissed their stale review August 24, 2026 14:27

A new stamphog review started for this PR — the fresh verdict replaces this approval.

stamphog[bot]
stamphog Bot previously approved these changes Aug 24, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contained dead-code removal within the owning team's own product area (frontend context field, backend freshness check, and the now-unreachable in-context formatting path in the Max tool); tests were reworked to match rather than deleted for coverage, and no risky territory (auth, billing, API contracts, CI, migrations) is touched.

  • Author wrote 9% of the modified lines and has 66 merged PRs in these paths (familiarity MODERATE).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 101L, 3F substantive, 237L/5F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1c-medium (237L, 5F, two-areas, chore)
stamphog 2.0.0b4 .stamphog/policy.yml @ 040c1d2 · reviewed head 040c1d2

1 updated
Run: 31a5f8c4-a37f-4934-be9e-0ebcc6d5bfa8

Co-authored-by: jurajmajerik <22996112+jurajmajerik@users.noreply.github.com>
@stamphog
stamphog Bot dismissed their stale review August 24, 2026 15:14

A new stamphog review started for this PR — the fresh verdict replaces this approval.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contained dead-code removal within the author's own team's product; tests updated to cover the surviving path and no lingering references to the removed code remain anywhere in the repo.

  • Author wrote 9% of the modified lines and has 308 merged PRs in these paths (familiarity MODERATE).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 103L, 4F substantive, 239L/6F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1c-medium (239L, 6F, two-areas, chore)
stamphog 2.0.0b4 .stamphog/policy.yml @ 5acdbbf · reviewed head 5acdbbf

@trunk-io
trunk-io Bot merged commit 28d899e into master Aug 24, 2026
273 checks passed
@trunk-io
trunk-io Bot deleted the experiments/max-summary-remove-dead-paths branch August 24, 2026 16:29
@deployment-status-posthog

deployment-status-posthog Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-08-24 17:08 UTC Run
prod-us ✅ Deployed 2026-08-24 17:47 UTC Run
prod-eu ✅ Deployed 2026-08-24 23:15 UTC Run

This branch was successfully deployed

1 active deployment
preview-pr-87799 — 5acdbbf2 Deployed Aug 24, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Request AI approval (no full review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant