Skip to content

perf(ci): scan only the formats a change set touches - #126727

Merged
trunk-io[bot] merged 2 commits into
masterfrom
perf/duplication-scan-touched-formats
Nov 2, 2026
Merged

perf(ci): scan only the formats a change set touches#126727
trunk-io[bot] merged 2 commits into
masterfrom
perf/duplication-scan-touched-formats

Conversation

@fuziontech

@fuziontech fuziontech commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Problem

CI contributors wait a median 5.2 minutes for the Duplication lint job on every PR that touches Python or TypeScript. Most of that is two jscpd scans over the whole repository even when the PR only touches one language.

Changes

  • The two lint scans now cover only the formats the change set touches. A Python-only PR stops parsing 24,589 TypeScript files; a TSX-only PR stops parsing 25,992 Python files. jscpd pairs files only within one format, so the omitted formats cannot hold a clone the branch introduced.
  • A change set that touches two or more scanned formats keeps the full scan, and so does one holding an extensionless file such as bin/sandbox, which jscpd reads by its shebang.
  • A change set with no scanned format, such as one that edits only this workflow or its limits file, skips both scans and passes in seconds.
  • Mechanical: write_findings now also writes the status file, so the early return leaves the same report files as a normal pass.

Measured

On this PR's own CI runs (Python-only change set, run 41183098839, run 41183466990 and run 41185469978): Lint for new duplication took 36s, 44s and 36s, and the whole job 86s, 96s and 86s. Baseline for the same step and job over the 37 other PR runs on this workflow in the same window (this PR's own runs excluded, so the baseline is untouched by the change): step median 258s, job (created to completed) median 313s.

Before (median, n=37) After (three runs)
Lint for new duplication step 258s 36s, 44s, 36s
Whole job 313s 86s, 96s, 86s

The scans did real work in all three: 8,863 clones found in two full Python scans (current tree and baseline), none new.

That is a much larger cut than the format ratio alone predicts, so the full three-format scan is likely superlinear in file count on a CI runner in a way a local 2-vCPU harness does not reproduce. Local end-to-end, same tree, worth stating separately:

Change set (local, 2 vCPU pinned) Before After
Python only 50.0s / 49.8s 40.5s / 40.5s
TSX only 50.6s / 49.7s 17.8s / 18.5s
New Python duplication (gate must fire) 49.7s / 50.0s, exit 1, one finding 41.1s / 40.9s, exit 1, same finding

How did you test this code?

Test rationale: the regression to catch is the gate going quiet on a real clone. test_scans_head_without_worktree_changes already guarded "the committed tree is what gets scanned"; it now also asserts the scan receives the narrowed format list, and returns a clone over the changed file so the baseline scan still has to run. TestFormatsTouched covers the mapping and the docs-only early return, including that the report files are still written. The duplication probe in the table above was run locally, to confirm the gate still fails on new duplication rather than trusting unit tests alone.

  • python -m unittest bin.test.test_lint_duplication: 49 tests pass.
  • ruff check and ruff format --check on both files.
  • Workflow linter: 14 checks pass across 146 workflows.
  • npx vitest run in tools/workflow-plan: 398 tests pass.
  • Not run here: the full dev stack, which cannot be built in this sandbox (uv sync fails building thrift on Python 3.14).

Release status

  • No feature flag controls this change

Docs update

  • No existing document covers the duplication lint internals.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Shelley, DeepSeek V4 Flash (deepseek-flash-api-deepseek-com)

Skills: /writing-tests, /writing-code-comments, /writing-pr-descriptions, /authoring-ci-workflows, /verifying-changes

More context for AI agents
  • Duplicate check: gh pr list --search "duplication in:title" found chore(ci): skip duplication baselines without candidates #126664, which skips the baseline scan when no clone pairs a changed file. This PR skips formats; the two are orthogonal and compose.
  • Decisions: multi-format change sets keep the full scan, because a two-format run measured no cheaper than the three-format one (19.0s against 16.2s), so narrowing there buys nothing.
  • Decisions: format extensions came from probes with npx jscpd@5.1.1: .d.ts, .mts and .cts land in typescript, .tsx is its own format, .pyx/.pxd/.pxi are python, .pyi is in none of the three, and extensionless shebang scripts are read as python.
  • Follow-ups: bin/test/test_lint_duplication.py runs in no CI job today, so gate changes are only verified locally; a step beside the other bin/test ones would close that.
  • Follow-ups: the CI number is one change set; the next step is comparing a week of step durations across the single-format and multi-format populations.
  • CI report: all sections ✅ except the Playwright flaky warning, which names Override/discard lifecycle from dashboard; this diff touches no frontend file.
  • Read closely: the early return in main, which must leave duplication-scan-status.json at ok or the poster reports a failed scan.
  • CI report: the universal-lane section comes from bin/** being mapped to UNIVERSAL in .github/scripts/trunk-impacted-targets.js. This diff does not change that mapping; narrowing it is a separate change with its own test surface.

@trunk-io

trunk-io Bot commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown

😎 Merged successfully - details.

@fuziontech
fuziontech requested a review from a team October 31, 2026 08:04
@parameterai

parameterai Bot commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown

Risk: No findings

This delta refactors the two jscpd scans into a shared scan_revision helper and skips the baseline scan when the current-tree scan already has no gate-failable clones, plus writes the scan-status file consistently. The logic stays fail-closed and conservative in every path (skipping the baseline can only keep more clones flagged, never fewer), and no security-relevant surface is touched.

Sentinel reviewed bc358b4 · Review settings

@github-actions

github-actions Bot commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🤖 CI report

🚨 Trunk lane — universal lane

This PR is assigned to the universal lane. It cannot merge in parallel with other PRs, so it can take longer to merge. Ask dev-ex if you think this is wrong.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️ Comment density — 5% of added code lines are comments (6 of 111)

This section warns when comments are more than 3% of the code lines a PR adds, and alerts above 6%. Before agent-assisted PRs, the typical share was about 2%. Only full-line comments count. Docstrings, generated files, snapshots, migrations, and workflow files are left out.

Comments that restate the code, record how the change came about, or narrate the next line add noise for the next reader. Keep the comments that explain a reason the code cannot show, and remove the rest. See .agents/skills/writing-code-comments/SKILL.md for the house rules.

Files with the most added comment lines:

File Comment lines Added lines
bin/test/test_lint_duplication.py 5 62
bin/lint_duplication.py 1 49

This check does not block merging. It updates on every push and clears when the share drops.

✅ Bundle size — 🟢 -1.3 KiB (-0.0%)

Uncompressed size of every built .js bundle, compared against the base branch.

Total: 76.15 MiB · 🟢 -1.3 KiB (-0.0%)

File Size Δ vs base
posthog-app/_parent/products/error_tracking/frontend/scenes/ErrorTrackingIssueScene/ErrorTrackingIssueScene.js 108.6 KiB 🟢 -1.3 KiB (-1.2%)

Posted automatically by build-bundle-size-report · uncompressed bytes from dist-report

✅ Eager graph — within budget

How much code each root ships on the eager path — downloaded and parsed before the surface is interactive. Measured from the esbuild output chunks (post-tree-shake, static imports only); lazy import() / React.lazy chunks are not counted.

Root Eager (shipped) Δ vs base Budget
entry (logged-out pages, app bootstrap)
src/index.tsx		 1.01 MiB · 25 files no change █████████░ 88.1% of 1.14 MiB
logged-out boot: index + App + bootApp (preloaded by every page, including /login)
src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts		 2.98 MiB · 626 files no change ██████████ 98.4% of 3.03 MiB
authenticated shell (every logged-in page)
src/scenes/AuthenticatedShell.tsx		 4.00 MiB · 1,275 files no change ██████████ 99.7% of 4.01 MiB
dashboard scene
src/scenes/dashboard/Dashboard.tsx		 9.44 MiB · 3,695 files no change ██████████ 96.0% of 9.84 MiB
today home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx		 4.92 MiB · 1,794 files no change ██████████ 99.7% of 4.93 MiB
AI-first home path
src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx		 6.17 MiB · 2,262 files no change ██████████ 98.4% of 6.27 MiB
events scene
src/scenes/activity/explore/EventsScene.tsx		 8.85 MiB · 3,417 files no change █████████░ 94.1% of 9.41 MiB
web analytics scene
src/scenes/web-analytics/WebAnalyticsScene.tsx		 9.65 MiB · 3,694 files no change █████████░ 92.0% of 10.49 MiB
replay detail scene
src/scenes/session-recordings/detail/SessionRecordingDetail.tsx		 7.37 MiB · 2,651 files no change █████████░ 90.7% of 8.13 MiB

🟢 node_modules/monaco-editor/ stays out of src/index.tsx
🟢 products/product_analytics/frontend/insights/activityDescriptions stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/index.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/index.tsx
🟢 node_modules/monaco-editor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/layout/navigation-3000/navigationLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/scenes/dashboard/dashboardLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/lemon-ui/LemonMarkdown/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/RichContentEditor/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/CodeSnippet/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/zod/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/logic/billing/billingLogic.tsx stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 src/lib/components/ProductEmptyState/productSetupPreloadLogic.ts stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/kea-test-utils/ stays out of src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
🟢 node_modules/monaco-editor/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/product_analytics/frontend/insights/activityDescriptions stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/dashboards/frontend/widgets/previews/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/player/sessionRecordingPlayerLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/panel-layout/ProjectTree/menus/SessionReplayMenuItems.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/taxonomy/core-filter-definitions-by-group.json stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/navigation/projectNoticeLogic.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/navigation/ImpersonationNotice/ImpersonationNotice.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/cohorts/cohortSavedToast.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/wizard/frontend/runs/WizardRunDetailsDrawer.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayShell.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayContextPanelFloating.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodaySidePanel.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/TodayTopBar.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/layout/today/todayRailItems.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 products/subscriptions/frontend/components/Subscriptions/exportNudge/exportNudgeLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/billing/paymentEntryLogic.ts stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@tiptap/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/prosemirror- stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/re2js/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/react-markdown/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/lib/components/Command/Command.tsx stays out of src/scenes/AuthenticatedShell.tsx
🟢 packages/quill/packages/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 zod/v4/locales/de.js stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/svg/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 node_modules/@posthog/brand/dist/generated/hoggies/components/ stays out of src/scenes/AuthenticatedShell.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/queries/validators.js stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/queries/web-analytics-validators.js stays out of src/scenes/dashboard/Dashboard.tsx
🟢 node_modules/re2js/ stays out of src/scenes/dashboard/Dashboard.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/dashboard/Dashboard.tsx
🟢 src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/scenes/project-homepage/today/TodayReportPage.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 src/queries/Query/Query.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 packages/quill/packages/charts/ stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
🟢 products/posthog_ai/frontend/components/tool/widgets/VisualizationWidget.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 products/posthog_ai/frontend/components/tool/widgets/RecordingsWidget.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 src/queries/Query/Query.tsx stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 products/posthog_ai/frontend/api/primitives.ts stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 src/scenes/notebooks/Notebook/notebookLogic.ts stays out of src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
🟢 src/scenes/session-recordings/playlist/SessionRecordingsPlaylist.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/web-analytics/tiles/WebAnalyticsTile.tsx stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/validators.js stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/queries/web-analytics-validators.js stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 node_modules/re2js/ stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/activity/explore/EventsScene.tsx
🟢 src/scenes/web-analytics/LiveMetricsDashboard/LiveWebAnalyticsMetrics.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 products/product_analytics/frontend/insights/trends/WorldMap/countryVectors.tsx stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/queries/validators.js stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 node_modules/re2js/ stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/web-analytics/WebAnalyticsScene.tsx
🟢 src/scenes/settings/SettingsMap.tsx stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 @jspm/core/nodelibs/browser/fs stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/re2js/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/kea-test-utils/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
🟢 node_modules/d3/src/ stays out of src/scenes/session-recordings/detail/SessionRecordingDetail.tsx

Largest files eagerly shipped from src/index.tsx
Size File
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
6.3 KiB ../node_modules/.pnpm/react@18.3.1/node_modules/react/cjs/react.production.min.js
4.7 KiB ../node_modules/.pnpm/@jspm+core@2.1.0/node_modules/@jspm/core/nodelibs/browser/process.js
4.0 KiB ../node_modules/.pnpm/scheduler@0.23.2/node_modules/scheduler/cjs/scheduler.production.min.js
1.9 KiB src/lib/utils/bootTiming.ts
1.7 KiB src/index.tsx
1.7 KiB ../node_modules/.pnpm/base64-js@1.5.1/node_modules/base64-js/index.js
1.2 KiB src/RootErrorBoundary.tsx
949 B src/scenes/ChunkLoadErrorBoundary.tsx
Largest files eagerly shipped from src/index.tsx + src/scenes/App.tsx + src/scenes/bootApp.ts
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
89.0 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
28.3 KiB src/lib/constants.tsx
24.6 KiB ../node_modules/.pnpm/buffer@6.0.3/node_modules/buffer/index.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
89.0 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
32.7 KiB src/lib/utils/eventUsageLogic.ts
28.6 KiB ../node_modules/.pnpm/zod@4.3.6/node_modules/zod/v4/core/schemas.js
Largest files eagerly shipped from src/scenes/dashboard/Dashboard.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
144.1 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
89.0 KiB src/lib/api.ts
84.9 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/surveys-preview.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/today/TodayHome.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
89.0 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
68.0 KiB ../products/posthog_ai/frontend/logics/runStreamLogic.ts
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
36.4 KiB ../node_modules/.pnpm/kea@4.0.0-pre.6_patch_hash=139b8d1f1304f9d9da452a9a1244c94ea679dbcb85687d8999563146879fb6f5_react@18.3.1/node_modules/kea/lib/index.cjs.js
Largest files eagerly shipped from src/scenes/AuthenticatedShell.tsx + src/scenes/project-homepage/ProjectHomepage.tsx + src/scenes/project-homepage/ai-first/AiFirstHomepage.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
144.1 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
89.0 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx
68.0 KiB ../products/posthog_ai/frontend/logics/runStreamLogic.ts
41.9 KiB ../node_modules/.pnpm/@dnd-kit+core@6.0.8_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@dnd-kit/core/dist/core.esm.js
Largest files eagerly shipped from src/scenes/activity/explore/EventsScene.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
144.1 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
89.0 KiB src/lib/api.ts
75.3 KiB src/scenes/dashboard/dashboardLogic.tsx
Largest files eagerly shipped from src/scenes/web-analytics/WebAnalyticsScene.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
144.1 KiB ../packages/quill/packages/primitives/dist/index.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
89.0 KiB src/lib/api.ts
84.9 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/surveys-preview.js
Largest files eagerly shipped from src/scenes/session-recordings/detail/SessionRecordingDetail.tsx
Size File
325.3 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/module.mjs
320.2 KiB ../node_modules/.pnpm/posthog-js@1.444.1_@types+react@18.3.27_react@18.3.1/node_modules/posthog-js/dist/rrweb.js
239.9 KiB ../packages/quill/packages/quill/dist/index.js
224.1 KiB ../node_modules/.pnpm/@posthog+icons@0.39.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js
126.8 KiB ../node_modules/.pnpm/react-dom@18.3.1_react@18.3.1/node_modules/react-dom/cjs/react-dom.production.min.js
104.2 KiB src/products.tsx
94.6 KiB ../node_modules/.pnpm/@tiptap+core@3.20.6_@tiptap+pm@3.20.6/node_modules/@tiptap/core/dist/index.js
92.6 KiB ../node_modules/.pnpm/prosemirror-view@1.40.1/node_modules/prosemirror-view/dist/index.js
89.0 KiB src/lib/api.ts
68.7 KiB src/lib/lemon-ui/icons/icons.tsx

Posted automatically by check-eager-graph · sizes are eager output bytes (shipped, post-tree-shake) from the esbuild metafile · part of #32479

✅ Frontend layers — no new upward imports, 0 removed

Static imports from a lower frontend layer into a higher one (layers, lowest first: kernel, shell, query, features). Compared with the base branch, so an import that a PR also adds to layer-baseline.txt still shows here. Sizes are the source bytes on the eager path of every logged-in page.

Layer Eager source bytes Δ vs base
kernel 2.96 MiB no change
shell 2.38 MiB no change
query 56.4 KiB no change
features 534.0 KiB no change

Download the interactive layer graph for this PR.

Posted automatically by check-layers

✅ Toolbar bundle — eager 2.34 MiB within budget

What the toolbar ships to customer pages, measured from the esbuild output (minified, post-tree-shake). The eager set is the entry plus everything statically imported from it — fetched before any feature runs; deferred chunks load lazily. The eager guardrail is 5.72 MiB. Each output file must also stay below 10 MB, where CloudFront stops compressing it. The module boundary is enforced separately by check-toolbar-graph.

Metric Size Δ vs base Budget
Eager (shipped)
entry + static imports		 2.34 MiB · 19 files no change ████░░░░░░ 40.9% of 5.72 MiB
Deferred (lazy) 2.11 MiB · 42 files no change n/a — loads on demand
Loader dist/toolbar.js 1.2 KiB no change █░░░░░░░░░ 6.0% of 19.5 KiB
Largest eagerly-shipped chunks
Size File
944.8 KiB dist/toolbar/toolbar-app-XM2XVS3V.css
696.2 KiB dist/toolbar/chunk-chunk-EKT3X5Y5.js
261.2 KiB dist/toolbar/chunk-chunk-JU6PYZZI.js
131.8 KiB dist/toolbar/chunk-chunk-FDH2IBXT.js
130.3 KiB dist/toolbar/chunk-chunk-KOKWFYFT.js
75.7 KiB dist/toolbar/toolbar-app-HU7UNWPX.js
69.0 KiB dist/toolbar/chunk-chunk-TSAL54PB.js
33.8 KiB dist/toolbar/chunk-chunk-F5VUPTPI.js
21.7 KiB dist/toolbar/chunk-chunk-U4EGNVMX.js
6.8 KiB dist/toolbar/chunk-chunk-DV7IWQNF.js

Posted automatically by check-toolbar-size · sizes are toolbar output bytes (shipped, post-tree-shake) from the esbuild metafile

✅ Dist folder size — 🟢 -11.9 KiB (-0.0%)

Total size of the built frontend/dist folder (all assets), compared against the base branch.

Total: 969.82 MiB · 🟢 -11.9 KiB (-0.0%)

⚠️ Playwright — 1 flaky

🎭 Playwright report · View test results →

⚠️ 1 flaky test:

  • password-protected insight sharing (chromium)

These issues are not necessarily caused by your changes.
Annoyed by this section? Help fix flakies and failures and it will go green!

@github-actions

github-actions Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Hey @fuziontech! 👋

It looks like your git author email on this PR isn't your @posthog.com address (fuziontech@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

  • Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
  • Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

posthog[bot]
posthog Bot reviewed Oct 31, 2026
View reviewed changes

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed ad2a5ed: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread bin/lint_duplication.py
touched = [
format_name
for format_name, extensions in FORMAT_EXTENSIONS.items()
if any(path.endswith(extensions) for path in changed_files)

@posthog posthog Bot Oct 31, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Include shebang-detected scripts when selecting scan formats

When a PR changes bin/sandbox alongside a .ts file, formats_touched selects only TypeScript. jscpd 5.1.1 also detects Python from shebangs, so the previous full scan covered this extensionless Python script, but the narrowed scan misses new clones it introduces. Account for shebang-detected files when selecting formats.

@hosthog

hosthog Bot commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown

HostHog preview — posthog-desktop-web

The previews for this PR have been torn down and no longer serve.

@fuziontech
fuziontech force-pushed the perf/duplication-scan-touched-formats branch from ad2a5ed to 9beaaf4 Compare October 31, 2026 08:09
@fuziontech @exedev-shelley
perf(ci): scan only the formats a change set touches
03ffce4 
The duplication gate runs two jscpd scans over the whole repository,
25,992 Python and 24,589 TypeScript files, on every PR that touches
either. jscpd pairs files only within one format, so a clone a branch
introduces always carries the format of a file that branch changed.
Scanning the other formats cannot change the verdict.

A change set that touches one format now scans that format alone. One
that touches two or more keeps the full scan, because a two-format run
measured no cheaper than the three-format one. So does a change set
holding an extensionless file, which jscpd reads by shebang and the
extension lists cannot place.

A change set with no scanned format in it, such as one that edits only
this workflow or its limits file, skips the scans entirely.

Co-authored-by: Shelley <shelley@exe.dev>
@fuziontech
fuziontech force-pushed the perf/duplication-scan-touched-formats branch from 9beaaf4 to 03ffce4 Compare October 31, 2026 08:39
fuziontech added a commit that referenced this pull request Nov 1, 2026
@fuziontech @exedev-shelley
fix(ci): drop unrelated duplication lint changes from branch
6457c1f 
Commit 03ffce4 was unintentionally bundled from branch perf/duplication-scan-touched-formats (PR #126727). Reverting it removes the duplication lint diff and resolves merge conflicts with master.

Co-authored-by: Shelley <shelley@exe.dev>
@fuziontech fuziontech mentioned this pull request Nov 1, 2026
Merged
gantoine
gantoine approved these changes Nov 2, 2026
View reviewed changes

@gantoine gantoine left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure why not 👍🏼

@fuziontech @exedev-shelley
chore(ci): merge master to resolve duplication scan conflicts
bc358b4 
Resolve the conflicts between the format-scoped scan and #126664's
skip-the-baseline optimisation. They compose: formats_touched() decides which
formats to scan, and scan_revision() now threads that list through so the
baseline scan is both narrowed and skipped when nothing fails.

The post-checkout-hook test needed a change in a scanned format, or the new
early-out returns before any worktree exists and the hook never fires.

Co-authored-by: Shelley <shelley@exe.dev>
@fuziontech

fuziontech commented Nov 2, 2026

Copy link
Copy Markdown
Member Author

/trunk merge

@trunk-io
trunk-io Bot merged commit 743f57b into master Nov 2, 2026
199 checks passed
@trunk-io
trunk-io Bot deleted the perf/duplication-scan-touched-formats branch November 2, 2026 21:35
@deployment-status-posthog

deployment-status-posthog Bot commented Nov 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-11-02 22:21 UTC Run
prod-us ✅ Deployed 2026-11-02 22:42 UTC Run
prod-eu ✅ Deployed 2026-11-02 22:43 UTC Run

@fuziontech

fuziontech commented Nov 3, 2026

Copy link
Copy Markdown
Member Author

Effectiveness, measured two nights after merge

Verdict: good. The job does what it promised.

Duplication lint job wall runs median range
before (merge at 2026-11-02T21:35:33Z) 6 202s 164-387s
after 7 103s 69-118s

49% faster, median 202s -> 103s. Sampling: measure.py attributes every sample by the workflow run's created_at against the merge time, not by the PR's merge date, and drops PRs whose real merged_at straddles the cutoff. Job durations come from the jobs API for this workflow (the job runs on Depot but the Duplication lint check is reported through the jobs API on this workflow, so no check-suite fallback was needed).

Affected PRs: before, 127714, 127712, 127697, 127606, 127604, 127600; after, 127943, 127939, 127919, 127910, 127907, 127898, 127858.

Caveat: 7 after-samples is a small population, and the two 199s runs in the before set are the modal case, so read the 49% as "roughly halves the job" rather than a precise number.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@posthog posthog[bot] posthog[bot] left review comments
@gantoine gantoine gantoine approved these changes

Assignees

@fuziontech fuziontech

Labels

team-devex

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@fuziontech @gantoine