Repository navigation
perf(ci): stop checking out full history to gate the nodejs suite - #126087
Conversation
The Node.js changes job checked out the whole repo with fetch-depth: 0 to run the paths filter, which reads the change set from the pull-request files API and never touches the tree. The checkout was 75s of an 80s job (median of 12 runs), and every job in the workflow needs that job. The job now checks out only the filter action, and the affected-crate detection that does need the merge base gets its own full-history checkout behind the rust filter that reaches it, with the same sparse cone ci-rust.yml gives the same binary. Co-authored-by: Shelley <shelley@exe.dev>
|
😎 Merged successfully - details. |
|
Hey @fuziontech! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
🤖 CI report✅ Trunk lane — non-backend laneThis PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes. |
HostHog preview —
|
|
Risk: No findings A CI-only performance change: the ci-nodejs gating job now sparse-checkouts only the paths-filter action instead of the full-history tree, and adds a conditional second full-history checkout (cone Sentinel reviewed |
Nightly Follow-Up ReviewReviewed the change against the workflow graph, security requirements, and real GitHub Actions run telemetry. 1. Claimed Performance Win ScrutinyRecomputed job and queue timing from the GitHub Actions API:
The numbers claimed in the PR description survive scrutiny: gating duration drops from ~85s to 6s, unblocking downstream jobs 78s earlier. 2. Workflow Correctness & Integrity
3. Security & Runner Policies
VerdictSafe to land. No bugs or regressions found. |
|
/trunk merge |
Effectiveness check: this workedVerified against the GitHub API on 2026-11-01, a day after this merged. Claim: the Measured:
89.5% faster, or ~77s off the gate on every Node-touching PR. The direction and the magnitude both match what the PR promised, and the gate is where the PR said it was: the pre-change job really did spend 84-99s, which is the full-history clone. How this was measured, since it matters for trusting the number: each sample is a merged PR's head SHA, and the run is picked by its Caveats: n is small (6 and 5), and the post-change spread reaches 21s on one run, which is runner variance rather than a regression. No revert needed. Posted by the nightly CI speed tuner. Method: check-runs API, run-side attribution by |
Problem
Determine need to run Node.js checksgate clones the repository's full history to decide what to run, and every job in the workflow needs that gate. The paths filter it runs reads the change set from the pull-request files API, so most of that clone is spent on history nothing in the job opens.Changes
rustfilter output that reaches it, with the same sparse coneci-rust.ymlalready gives the same binary./authoring-ci-workflowsrecording the two-step pattern, since the section on gating jobs recommends the cheap checkout but says nothing about the step that needs history.The step graph inside the gate job, before and after:
Measured from the Actions API, on real runs:
Determine need to run Node.js checks, job durationThe lane end is set by the suite, not by the gate, so a run saves the gate delta when its jobs take the same time: 74s of a ~7.5 min lane, about 16%. On this PR's own run the Rust ingestion e2e job happened to take 417s against 326s in the before sample, which hides the 74s in the lane-end number, so the table reports the two figures that isolate the change instead.
How did you test this code?
Test rationale: no new test. A workflow edit is verified by the runs it produces, not by a unit test, and the workflow planner suite already pins which jobs run in each scenario.
PYTHONPATH=tools/hogli-commands:tools/hogli/src python3 -c "..."— all 14 workflow lint checks pass.cd tools/workflow-plan && npx vitest run— 393 tests pass; the ci-nodejs plan output is unchanged in every scenario, since noif:moved.node tools/workflow-plan/src/cli.ts .github/workflows/ci-nodejs.yml— drafts, ready, fork, queue and master rows unchanged.rust/orproto/file), so its checkout was verified locally instead: a--depth=1 --filter=blob:noneclone, unshallowed with--filter=blob:none, switched to therust/ proto/ .github/cone, and the base-commit worktree thataffected-servicesbuilds — the three git operations the step performs — all succeed, andgit diff --name-only <base>...HEADreturns the change set the binary reads. It also matches the checkoutci-rust.yml'saffectedjob already runs in production, and a wrong cone fails the step loudly rather than silently skipping the suite.Release status
Docs update
/authoring-ci-workflowsnote in Changes; the existing checkout section already says a gating job should sparse-checkout the filter action.🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Agent: Shelley (
shelley), modeldeepseek-flash-api-deepseek-com(DeepSeek's own platform API)Skills:
/authoring-ci-workflows,/writing-pr-descriptionsMore context for AI agents
changes+affected+decidejobs, so noif:moved and the planner expectations are untouched..github/ proto/ rust/— a superset of whatci-rust.ymlgives the binary, so a missed path is unlikely rather than merely undetected.ci-nodejs.ymlalone; nothing else claims this job.Determine need to run frontend checksshows a 300s outlier in the same sample (median 10s); worth a look if it repeats.if:and cone inci-nodejs.yml; it is the only behavior change for Rust-touching PRs.