Skip to content

perf(ci): size a narrowed Storybook run from the timing manifest - #127960

Merged
trunk-io[bot] merged 4 commits into
masterfrom
perf/ci-size-storybook-shards-from-the-manifest
Nov 4, 2026
Merged

trunk-io[bot] merged 4 commits into
masterfrom
perf/ci-size-storybook-shards-from-the-manifest

Conversation

@fuziontech

@fuziontech fuziontech commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

Problem

select-stories sizes the narrowed Storybook matrix from a flat 6 seconds per affected story:

const suggestedShards = Math.max(1, Math.floor((affectedDuration * 2) / TARGET_SHARD_SECONDS) + 1)

The stories a frontend diff actually reaches are not average stories. Replaying the selector against the storybook-module-graph artifact of four recent runs, the affected set carries 33 to 41 seconds of render time per story, against 22.0 seconds for the manifest mean. Sizing at 6 seconds undercounts the work by five to six times.

So the narrowed run became the slowest thing in CI. Measured per-shard walls from narrow runs:

Run PR Shards Affected stories Median shard wall
41528652788 #127833 4 161 911s
41557823345 #127858 4 189 938s
41537473253 #127857 8 372 874s
41520310636 #127778 8 372 885s

A full 28-shard run finishes its shards in 378s. Storybook was ending the lane on 9 of 25 sampled merged PRs, precisely because the "narrowed" path was slower than the full one.

Changes

  • bin/find-affected-stories reads common/storybook/storybook-timings.json and sums real render seconds for the affected set, instead of count * 6. That is the manifest common/storybook/test-sequencer.js already balances shards with, so the selector and the sequencer now agree on what a story costs.
  • suggestShards holds the shard density of the full matrix rather than a fixed 10-minute target: ceil(affected_seconds / total_seconds * full_shards). A narrowed run then finishes its tests in about the wall time a full run does.
  • select-stories reads the manifest from the trusted base checkout it already uses for the selector (one more file in that sparse checkout) and passes --timings and --max-shards "$CHROMIUM_SHARDS". The base copy is the one the nightly refresh updates, so a branch behind master is not sized from a stale manifest, and a fork cannot steer its shard count. --max-shards keeps the count tied to the matrix in ci-storybook.yml rather than a second constant, and a non-numeric value falls back to the full-matrix default instead of emitting NaN into the matrix.
  • The selector prints how many affected stories the manifest timed (Timed N of M affected stories ...), so a key-format drift that silently sends every story to the median is visible in the job log.
  • The job summary gains an "Est. shard wall" row. The existing "Est. CI minutes" row keeps its runner-minute meaning; only its inputs got more accurate.

No change to which stories run. A narrowed run still runs every story the module graph reached, on the same browser, with the same retries.

Measured

The four runs, replayed with the new selector. The shard wall holds the shard's own fixed startup (median 138.5s: container init, checkout, pnpm restore, test-runner boot) constant and scales only the test phase, which is what actually moves with shard count.

Run Shards Median shard wall Storybook lane PR lane
41528652788 4 → 8 911s → 525s 25.2 → 18.8 min 25.2 → 19.3 min
41557823345 4 → 9 938s → 494s 22.1 → 14.7 min 22.1 → 17.5 min
41537473253 8 → 15 874s → 531s 20.9 → 15.2 min 20.9 → 15.2 min
41520310636 8 → 15 885s → 537s 22.8 → 17.0 min 22.8 → 17.0 min
median 898s → 528s (-41%) 22.5 → 16.1 min (-28%) 22.45 → 17.25 min (-23%)

The PR lane is the later of Storybook and the run's next-longest workflow, so two of the four are now ended by another lane: #127858 by E2E at 17.5 min, #127833 by E2E at 19.3 min. E2E at 17.5 to 19.3 min is the floor after this.

Method

This is a replay, not a production A/B. For each run I downloaded that run's own storybook-module-graph artifact, checked out the run's head SHA in a worktree, and ran the new selector against it, so the affected set and the story seconds are the ones CI really saw. The shard wall then scales as old_test * old_shards / new_shards.

The model is calibrated against full runs: 18,911 story-seconds over 28 shards and 2 workers predicts a 338s test phase against 378s observed, so it is about 12% optimistic. On the four replays it lands within 3% of the observed test phase (799s observed against 774s predicted for run 41557823345). Walls and lanes come from the runs' own jobs; the lane is the run's created_at to its last Storybook check completing.

Runner cost

This trades compute for wall-clock on the narrowed path, which is the honest cost. Shard runner-seconds before and after:

Run Before After Delta
41528652788 3,644s 4,198s +9.2 min
41557823345 3,750s 4,442s +11.5 min
41537473253 6,992s 7,962s +16.2 min
41520310636 7,080s 8,049s +16.2 min

A narrowed run costs 9 to 16 more runner-minutes to save 5.7 to 7.4 minutes of wall time. Narrow runs are roughly 1 in 7 PR runs, and full-matrix runs are untouched. The trade reverses the current shape: today a narrow run burns fewer runner-minutes to produce the slowest lane in CI.

How did you test this code?

Test rationale: the sizing function is the whole change, and it is pure arithmetic a reviewer can get wrong with floor where ceil is needed, or by dropping the cap. suggestShards is exported, and bin/find-affected-stories.test.mjs pins the bug this PR fixes rather than the implementation: half the story seconds must give half the shards (5000/10000/28 -> 14), a change set that touches every story must not exceed the full matrix, and a small set must still ask for more than one shard. The four existing selection tests still pass.

The test itself was dead code: bin/find-affected-stories.test.mjs was added with the selector in #69932 and no workflow has ever run it. ci-scripts.yml now runs it beside the other script tests and triggers on the selector, so the sizing arithmetic gates future changes rather than sitting in a file nobody reads.

--help documents the new flags, and --max-shards falls back to the full-matrix default on a bad value. loadTimings (bad or missing files) and workloadSeconds (median for untimed stories, flat fallback without a manifest) have their own tests.

CI on this branch will not show the new sizing: select-stories runs the selector from the base branch (story-selector/bin/find-affected-stories) on purpose, so a PR cannot choose its own test selection. That base copy predates --timings and --max-shards, so this branch's own Storybook run rejects the flags, hits the step's existing continue-on-error, and falls back to the full 28-shard matrix. That is the intended degraded path and it stays green; it costs one full run on this PR only. The change takes effect on the first PR after merge, and Visual regression tests - chromium (1) on narrowed runs is where to check it.

CI report

The bundle-size section reports +71 B (+0.0%). This diff contains no file under frontend/, products/*/frontend/ or packages/, so that delta is base-branch drift between the merge commit and the last build of master, not a byte this PR adds. Trunk lane: universal, because the diff edits workflow files.

Release status

  • No feature flag controls this change

Docs update

  • docs/internal/ci-things-already-tried.md records that the trusted base checkout in select-stories now also carries the timing manifest, and why a story count cannot size a shard count.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Shelley, DeepSeek V4.1 Flash (deepseek-flash-api-deepseek-com)

Skills: /authoring-ci-workflows, /writing-pr-descriptions, /writing-tests, /writing-code-comments

More context for AI agents
  • Session: nightly CI speed run for PostHog/posthog.
  • Decisions: reuse common/storybook/storybook-timings.json rather than add a timing source, read from the base checkout next to the selector rather than the PR head. A PR-head copy could only steer its own run between 1 and CHROMIUM_SHARDS shards, but the base copy keeps the job's "selector inputs come from base" rule whole and is never stale.
  • Decisions: density-preserving sizing over a fixed wall target. A 10-minute target is what let a 4-shard run reach 911s of test wall.
  • Related: ci(playwright): run queue gate jobs on depot runners #125810 and perf(ci): stop checking out full history to gate the nodejs suite #126087 moved the gate jobs off GitHub-hosted runners. Different lever: this is the shard count a narrowed run asks for.
  • Read closely: the suggestShards cap, and CHROMIUM_SHARDS in ci-storybook.yml, which is the only place the full-matrix size is defined.
  • Follow-up: after merge, measure narrowed-run Visual regression tests - chromium (1) walls. The 12% optimism in the model means a run may land above its projected wall.

A selective Storybook run got shards from a flat 6s per story, while the
stories a frontend diff affects really take about 40s each. A run that
selected 155 to 190 stories landed on 4 shards and 858s to 1124s of test
wall per shard; the 28-shard full matrix finishes its shards in 378s.

Size the shard count from common/storybook/storybook-timings.json, the
manifest the test sequencer already balances with, and hold the shard
density of the full matrix instead of a fixed 10-minute target.

Co-authored-by: Shelley <shelley@exe.dev>
@fuziontech
fuziontech requested a review from a team November 3, 2026 06:55
@trunk-io

trunk-io Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

@fuziontech fuziontech self-assigned this Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Hey @fuziontech! 👋

It looks like your git author email on this PR isn't your @posthog.com address (fuziontech@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

@parameterai

parameterai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Risk: No findings

This delta moves the story timing manifest from the PR head checkout to the trusted base-branch checkout (story-selector/), so a fork can no longer steer its own shard sizing through the manifest, and adds a stderr hit-rate log plus unit tests for the manifest loader and workload costing. I found no security issues in the changed lines: the manifest source is now maintainer/base-controlled, CHROMIUM_SHARDS is a workflow constant, the new log line interpolates only counts and a workflow-fixed path, and shard counts stay clamped to 1..max.

Sentinel reviewed a897603 · Review settings

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

🚨 Trunk lane — universal lane

This PR is assigned to the universal lane. It cannot merge in parallel with other PRs, so it can take longer to merge. Ask dev-ex if you think this is wrong.

⚠️ Bundle size — 🔺 +284.5 KiB (+0.4%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 76.78 MiB · 🔺 +284.5 KiB (+0.4%)

File Size Δ vs base
exporter/src/exporter/scenes/ExporterNotebookScene.js 4.04 MiB 🔺 +109.7 KiB (+2.7%)
posthog-app/_parent/products/ai_observability/frontend/AIObservabilityTraceScene.js 141.4 KiB 🟢 -88.2 KiB (-38.4%)
render-query/src/render-query/render-query.js 21.14 MiB 🔺 +85.3 KiB (+0.4%)
posthog-app/_parent/products/product_analytics/frontend/insights/explainChart/InsightExplainChart.js 38.6 KiB 🔺 +38.6 KiB (new)
exporter/_parent/products/product_analytics/frontend/insights/explainChart/InsightExplainChart.js 34.1 KiB 🔺 +34.1 KiB (new)
posthog-app/_parent/products/slack_app/frontend/workspace-setup/SlackAppWorkspaceSetupScene.js 12.4 KiB 🔺 +12.4 KiB (new)
exporter/src/queries/nodes/InsightViz/InsightViz.js 376.5 KiB 🔺 +11.3 KiB (+3.1%)
posthog-app/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartButton.js 9.0 KiB 🔺 +9.0 KiB (new)
posthog-app/_parent/products/data_catalog/frontend/DataCatalogScene.js 46.3 KiB 🔺 +9.0 KiB (+24.1%)
posthog-app/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartAgentBridge.js 8.6 KiB 🔺 +8.6 KiB (new)
posthog-app/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartAIAnalysisFallback.js 8.3 KiB 🔺 +8.3 KiB (new)
exporter/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartButton.js 6.5 KiB 🔺 +6.5 KiB (new)
exporter/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartAgentBridge.js 6.1 KiB 🔺 +6.1 KiB (new)
exporter/_parent/products/product_analytics/frontend/insights/explainChart/ExplainChartAIAnalysisFallback.js 5.9 KiB 🔺 +5.9 KiB (new)
posthog-app/src/lib/components/CommandKSearch/CommandKSearch.js 58.1 KiB 🔺 +5.6 KiB (+10.7%)
posthog-app/_parent/products/data_modeling/frontend/ModelsScene.js 91.7 KiB 🔺 +2.1 KiB (+2.4%)
posthog-app/_parent/products/slack_app/frontend/workspace-setup/offerSlackWorkspaceSetup.js 2.0 KiB 🔺 +2.0 KiB (new)
exporter/_parent/products/slack_app/frontend/workspace-setup/offerSlackWorkspaceSetup.js 1.7 KiB 🔺 +1.7 KiB (new)
exporter/_parent/products/workflows/frontend/Workflows/Workflow.js 236.2 KiB 🔺 +1.6 KiB (+0.7%)
posthog-app/_parent/products/ai_observability/frontend/clusters/AIObservabilityClusterScene.js 28.0 KiB 🔺 +1.3 KiB (+5.1%)
posthog-app/_parent/products/ai_observability/frontend/AIObservabilityScene.js 173.2 KiB 🔺 +1.3 KiB (+0.7%)
exporter/src/queries/schema.js 1.29 MiB 🔺 +1.1 KiB (+0.1%)
posthog-app/src/queries/schema.js 1.29 MiB 🔺 +1.1 KiB (+0.1%)
posthog-app/_parent/products/signals/frontend/inbox/InboxScene.js 503.0 KiB 🟢 -1.1 KiB (-0.2%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx
1.01 MiB · 25 files 🔺 +5.0 KiB (+0.5%) █████████░ 88.7% of 1.14 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login)
src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
2.99 MiB · 623 files 🔺 +38.3 KiB (+1.3%) ██████████ 98.7% of 3.03 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx
3.87 MiB · 1,238 files 🔺 +36.3 KiB (+0.9%) ██████████ 96.4% of 4.01 MiB
dashboard scene
src/scenes/dashboard/Dashboard.tsx
8.31 MiB · 3,041 files 🔺 +53.0 KiB (+0.6%) ████████░░ 84.5% of 9.84 MiB
today home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
4.71 MiB · 1,762 files 🔺 +53.3 KiB (+1.1%) ██████████ 95.6% of 4.93 MiB
AI-first home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
5.95 MiB · 2,260 files 🔺 +55.1 KiB (+0.9%) █████████░ 94.9% of 6.27 MiB
events scene
src/scenes/activity/explore/EventsScene.tsx
7.35 MiB · 2,626 files 🔺 +39.0 KiB (+0.5%) ████████░░ 78.1% of 9.41 MiB
web analytics scene
src/scenes/web-analytics/WebAnalyticsScene.tsx
8.26 MiB · 2,951 files 🔺 +68.9 KiB (+0.8%) ████████░░ 78.7% of 10.49 MiB
replay detail scene
src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
7.41 MiB · 2,654 files 🔺 +50.5 KiB (+0.7%) █████████░ 91.2% of 8.13 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 products/product_analytics/frontend/insights/activityDescriptions stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/zod/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/logic/billing/billingLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/ProductEmptyState/productSetupPreloadLogic.ts stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/kea-test-utils/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/product_analytics/frontend/insights/activityDescriptions stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/dashboards/frontend/widgets/previews/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/panel-layout/ProjectTree/menus/SessionReplayMenuItems.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/navigation/projectNoticeLogic.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/navigation/ImpersonationNotice/ImpersonationNotice.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/cohorts/cohortSavedToast.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/wizard/frontend/runs/WizardRunDetailsDrawer.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayShell.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayContextPanelFloating.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodaySidePanel.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayTopBar.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/todayRailItems.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/subscriptions/frontend/components/Subscriptions/exportNudge/exportNudgeLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/billing/paymentEntryLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/conversations/frontend/components/SidePanel/sidepanelTicketsLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/zod/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/authentication/auth/loginLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/authentication/shared/passkeyLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/surveys/constants.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@tiptap/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/prosemirror- stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/re2js/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/react-markdown/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/Command/Command.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 packages/quill/packages/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 zod/v4/locales/de.js stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/queries/validators.js stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/queries/web-analytics-validators.js stays out of src/scenes/dashboard/Dashboard.tsx
🟢 node_modules/re2js/ stays out of src/scenes/dashboard/Dashboard.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/debug/HogDebug.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/queries/QueryEditor/QueryEditor.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/project-homepage/today/TodayReportPage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/queries/Query/Query.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 packages/quill/packages/charts/ stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 products/posthog_ai/frontend/components/tool/widgets/RecordingsWidget.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 src/queries/Query/Query.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 products/posthog_ai/frontend/api/primitives.ts stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 src/scenes/notebooks/Notebook/notebookLogic.ts stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/validators.js stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/web-analytics-validators.js stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 node_modules/re2js/ stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/nodes/InsightViz/InsightViz.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/nodes/DataVisualization/DataVisualization.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/nodes/WebOverview/WebOverview.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/debug/HogDebug.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/QueryEditor/QueryEditor.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/scenes/web-analytics/LiveMetricsDashboard/LiveWebAnalyticsMetrics.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 products/product_analytics/frontend/insights/trends/WorldMap/countryVectors.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/queries/validators.js stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 node_modules/re2js/ stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/scenes/debug/HogDebug.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/queries/QueryEditor/QueryEditor.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 packages/quill/packages/quill/dist/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 src/scenes/settings/SettingsMap.tsx stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 @jspm/core/nodelibs/browser/fs stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/re2js/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/d3/src/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.7 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
4.0 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.9 KiB src/lib/utils/bootTiming.ts
1.7 KiB src/index.tsx
1.7 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.2 KiB src/RootErrorBoundary.tsx
1.1 KiB src/lib/utils/deferredAppContext.ts
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.4 KiB src/products.tsx
88.4 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
28.4 KiB src/lib/constants.tsx
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.4 KiB src/products.tsx
88.4 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
32.7 KiB src/lib/utils/eventUsageLogic.ts
28.4 KiB src/lib/constants.tsx
Largest files eagerly shipped from src/scenes/dashboard/Dashboard.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
114.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/surveys-preview.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.4 KiB ../packages/quill/packages/components/dist/index.js
88.4 KiB src/lib/api.ts
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
88.4 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
68.0 KiB ../products/posthog_ai/frontend/logics/runStreamLogic.ts
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
88.4 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
68.0 KiB ../products/posthog_ai/frontend/logics/runStreamLogic.ts
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
41.7 KiB src/scenes/hog-functions/sub-templates/sub-templates.ts
Largest files eagerly shipped from src/scenes/activity/explore/EventsScene.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.4 KiB ../packages/quill/packages/components/dist/index.js
88.4 KiB src/lib/api.ts
75.3 KiB src/scenes/dashboard/dashboardLogic.tsx
Largest files eagerly shipped from src/scenes/web-analytics/WebAnalyticsScene.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
114.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/surveys-preview.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.4 KiB ../packages/quill/packages/components/dist/index.js
88.4 KiB src/lib/api.ts
Largest files eagerly shipped from src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
Size File
355.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
321.5 KiB ../node_modules/.pnpm/posthog-js@1.444.4_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/rrweb.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
108.9 KiB ../packages/quill/packages/primitives/dist/index.js
104.4 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
90.4 KiB ../packages/quill/packages/components/dist/index.js
88.4 KiB src/lib/api.ts

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Frontend layers — no new upward imports, 3 removed

Static imports from a lower frontend layer into a higher one (layers, lowest first: kernel, shell, query, features). Compared with the base branch, so an import that a PR also adds to layer-baseline.txt still shows here. Sizes are the source bytes on the eager path of every logged-in page.

Layer Eager source bytes Δ vs base
kernel 2.97 MiB 🔺 +2.5 KiB (+0.1%)
shell 2.39 MiB 🔺 +2.4 KiB (+0.1%)
query 56.4 KiB no change
features 379.1 KiB 🔺 +676 B (+0.2%)

🟢 3 upward import(s) removed. Run hogli lint:frontend-layers --prune-baseline if CI asks.

Download the interactive layer graph for this PR.

Posted automatically by check-layers

✅ Toolbar bundle — eager 2.37 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports
2.37 MiB · 19 files 🔺 +35.7 KiB (+1.5%) ████░░░░░░ 41.5% of 5.72 MiB
Deferred (lazy) 2.11 MiB · 42 files no change n/a — loads on demand
Loader dist/toolbar.js 1.2 KiB no change █░░░░░░░░░ 6.0% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
950.3 KiB dist/toolbar/toolbar-app-IQE5CVI6.css
727.0 KiB dist/toolbar/chunk-chunk-VDG6TBHG.js
261.2 KiB dist/toolbar/chunk-chunk-HOBPFMJK.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
130.3 KiB dist/toolbar/chunk-chunk-YRKTAGNC.js
75.7 KiB dist/toolbar/toolbar-app-VOIAJ6KC.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
33.8 KiB dist/toolbar/chunk-chunk-DB2ELZ3J.js
21.7 KiB dist/toolbar/chunk-chunk-QR66VVGM.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🔺 +3.75 MiB (+0.4%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 970.85 MiB · 🔺 +3.75 MiB (+0.4%)

✅ Playwright — all passed

All tests passed.

View test results →

fuziontech and others added 3 commits November 3, 2026 06:56
bin/find-affected-stories.test.mjs has run in no workflow since it was
added. It now runs in ci-scripts.yml beside the other script tests, and
ci-scripts.yml triggers on the selector so a change to it runs its test.

Co-authored-by: Shelley <shelley@exe.dev>
The job uses a sparse checkout, so bin/find-affected-stories was absent
and the new step failed on a missing file rather than on a test.

Co-authored-by: Shelley <shelley@exe.dev>
The selector already runs from a base-branch checkout so a PR cannot change
how it narrows. The manifest that sizes the shard count came from the PR head,
which a fork controls and which goes stale on a branch behind master. It now
comes from the base checkout next to the selector, where the nightly refresh
lands.

The selector also prints how many affected stories the manifest timed, so a
key format drift that silently sends every story to the median shows in the
job log, and loadTimings/workloadSeconds get tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fuziontech

Copy link
Copy Markdown
Member Author

Review follow-up in a897603a9

This PR had no reviews, so I gave it a fresh read before anyone else does. The sizing logic holds up:

  • Manifest keys and module-graph names are both repo-relative, so lookups line up (858 timed stories, median 15.9s).
  • The shard count is capped at CHROMIUM_SHARDS both in the script and again in jq.
  • A missing or malformed manifest falls back to the flat estimate.

Two changes came out of the review.

  • The manifest now comes from the trusted base checkout, not the PR head. select-stories runs the selector from a base-branch checkout precisely so a PR cannot steer its own narrowing. Reading the sizing input from the PR head was an exception to that rule. A fork could only move its own shard count between 1 and 28, but the head copy also goes stale on a branch behind master, while the base copy is the one the nightly refresh updates. The head checkout is back to .nvmrc only. common/storybook/storybook-timings.json joins bin/find-affected-stories in the story-selector sparse checkout, and --timings points there. docs/internal/ci-things-already-tried.md and the PR body are updated to match.
  • A silent fallback is now visible, and there are tests for it. If the manifest keys ever drift from the graph's names, every story quietly costs the median and nothing fails. The selector now logs Timed N of M affected stories from <path>; the rest cost the median. loadTimings (bad or missing files) and workloadSeconds (the median for untimed stories, the flat fallback without a manifest) are exported and tested alongside suggestShards.

As the PR body already says, this branch's own Storybook run can't exercise the new sizing, because the base selector predates the flags. The first narrowed run after merge is the real check. Look for the Timed N of M line in Story selection next to the Visual regression tests - chromium shard walls.

@fuziontech
fuziontech requested a review from gantoine November 3, 2026 16:03
@fuziontech

Copy link
Copy Markdown
Member Author

/trunk merge

@trunk-io
trunk-io Bot merged commit 59e1228 into master Nov 4, 2026
262 checks passed
@trunk-io
trunk-io Bot deleted the perf/ci-size-storybook-shards-from-the-manifest branch November 4, 2026 07:18
@deployment-status-posthog

deployment-status-posthog Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-11-04 07:35 UTC Run
prod-us ✅ Deployed 2026-11-04 07:50 UTC Run
prod-eu ✅ Deployed 2026-11-04 07:47 UTC Run

@fuziontech

Copy link
Copy Markdown
Member Author

Effectiveness, measured 2026-11-05. It did what it promised. Verdict: good.

Storybook, full run before after
chromium shard wall, median 872s 426s (-51%)
Storybook lane, median 23.3 min 13.6 min (-42%)
shards per narrowed run, median 8 26

Method. Attribution is by each workflow run's created_at against the merge time, never by the PR's merge date, because this repo re-runs old PRs through the merge queue. Before: the 10 most recent merged PRs whose last pre-merge Storybook run succeeded, merged 2026-10-27 to 2026-11-02. After: 27 runs created after the 2026-11-04T07:18:24Z merge. Shard durations come from each run's check suite, since Depot jobs do not appear in the jobs API. Shard counts come from the check names.

The lane figure uses 20 of the 27 after-runs. Seven are excluded because their lane ran 31 to 72 minutes, which is a runner queue wait on the workflow as a whole, not shard work; the shard wall is unaffected by that exclusion (408s over the 20).

The replayed estimate in the description (898s to 528s per shard) was 15% pessimistic against what production did (872s to 426s), because the real selector lands nearer the full-matrix density than the replay modelled.

Two PRs for the record, both measured in the same window: #127833 and #127858, which the description named as the runs sized at 4 shards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants