Skip to content

ci: move PR analysis and Linux unit tests to Blacksmith - #29362

Merged
nwparker merged 15 commits into
mainfrom
nwparker/ci-blacksmith-linux-capacity
Oct 16, 2026
Merged

nwparker merged 15 commits into
mainfrom
nwparker/ci-blacksmith-linux-capacity

Conversation

@nwparker

@nwparker nwparker commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

ELI5

PRs wait for busy GitHub runners even when checks themselves take only minutes. Move the PR detector, lint/typecheck job, five Linux unit-test shards, and final result to Blacksmith. Keep Linux packaging, relay, Windows, compatibility checks, and other workflows on GitHub.

What Changed

  • Route detection to Blacksmith Ubuntu 24.04 x64 with 2 CPUs, lint/typecheck to x64 with 8 CPUs/32 GB, unit shards to x64 with 4 CPUs/16 GB, and the aggregate to the existing Blacksmith Ubuntu 22.04 2-CPU label.
  • Make the scanner and hook test fixtures explicitly change the metadata used to invalidate their caches, avoiding same-clock-tick assumptions exposed by the faster runners. The daemon fixture waits for ctime to advance and asserts it changed while mtime stays restored. Maintenance tests advance fake time in small ticks so pending real I/O does not exhaust the attempt deadline. Existing result assertions and production cache behavior stay intact.
  • Retain five shards and four workers per migrated shard. Keep the existing checks, conditions, timeouts, and aggregate failure handling.
  • Preserve an explicitly supplied HOME in the ephemeral runner's /etc/profile.d/blacksmith.sh; the stock script resets it to /home/runner. Verify Electron startup and an actual login-shell fixture before starting the suite. Fail early if either contract is missing. No tests are skipped or assertions weakened.
  • Reuse the existing Linux cache-preparation job as a matrix: retain GitHub ARM cache production and add Blacksmith x64 production using the existing dependency/native actions.

Why

On #29246, the successful commit took 107m37s to reach the required gate: about 95m15s waiting for runners and 12m22s executing its critical path. The five-second aggregate alone waited 17m21s. The daily demand report estimates 548 PR-check runner-hours and 454 GitHub ARM runner-hours across the repository for the sampled day.

Target the repeated Linux work instead of moving every workflow or increasing shard fan-out. The ARM Blacksmith qualification started jobs within 10–14 seconds but shards ran about 15 minutes and exposed missing Electron libraries, shell-profile differences, and timing failures. Qualify x64 instead: Blacksmith documents its x64 image as following GitHub's x64 image and using gaming CPUs. The 8-CPU preflight leaves memory for overlapping compiler checks; unit shards retain four workers.

Published Linux x64 rates are $0.004/min for 2 CPUs, $0.008/min for 4 CPUs, and $0.016/min for 8 CPUs. On the final commit, all five unit shards passed in 6m30s–6m41s, with 10–11 seconds between admission and runner start. Preflight passed in 1m38s after an 11-second pickup. With minute rounding, the migrated jobs cost approximately $0.32 per full run before any free allowance, excluding retained GitHub jobs. This is usage-based spend, not a configured budget cap. These are observed runs, not a controlled same-commit benchmark against GitHub.

Linked Issue

Maintainer CI maintenance following the backlog observed in #29246; related #29337. No new issue needed.

Visual Proof

N/A — CI runner routing only; no application UI or behavior change.

Testing

  • I manually tested these changes locally
  • Automated tests added/updated, or explained why not below

461 tests pass across fourteen workflow/cache/aggregate/admission/lint-scope and affected scanner/hook suites. The full anti-pattern scan also passes locally. Changed-code quality, formatting, precommit lint, and git diff --check pass. The worker regression evaluates both GitHub/Blacksmith ARM and Blacksmith x64, and confirms other x64 callers retain their default. On final head d240809f16126733a8931ac153aa1329125cc7fc, hosted qualification has passed preflight and all five unit shards. All five shards passed 120,714 tests. Retained GitHub checks and the full aggregate remain pending. The repository owner explicitly instructed merging immediately and skipping pending CI because the queue backlog is the problem this migration addresses; this is an exception to the normal exact-head aggregate requirement.

Review

Reuses documented runner labels and existing dependency/native actions. Cache keys still include architecture and native-runtime identity. The omitted Linux warmers' installation and Electron preparation run in the real PR consumers; Windows warmer validation remains separate. Architecture-specific and release checks retain their existing runners. No application behavior or wire protocol changes. The same-size/preserved-mtime generation suite also passes, preserving coverage beyond the updated fixtures. The two later filesystem/timer failures passed ten repeated focused runs after fixing their fixture timing.

Agent skill upstream boundary

  • Not applicable; no upstream skill source copied.

Notes

Existing queued runs keep their original workflow; routing takes effect for new runs after merging. This change does not reroute already queued work. Nine stale closed/merged-PR runs were separately requested to cancel after rechecking their state and identity; a still-active merged-PR run was force-cancelled after normal cancellation did not release it.

Checklist

  • This PR is small and focused
  • Explained user experience, mechanism, and choice of approach
  • N/A visual proof explained
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH, and folder workspace impact considered: CI only
  • Full PR workflow and required aggregate pass at the exact final head — explicitly waived by the repository owner for this merge; preflight and all five unit shards passed.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 24b8159b-34e4-47cd-b739-32dbf064632a
📥 Commits

Reviewing files that changed from the base of the PR and between a15cd22 and d240809.

📒 Files selected for processing: 3
  • src/main/daemon/daemon-host-file-hash-cache.test.ts
  • src/main/linear/client.test.ts
  • src/shared/repo-ref-maintenance.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

CI workflows move several jobs to Blacksmith runners, add a Linux runtime warmup matrix, and adjust unit-test runner setup and worker limits. Workflow tests check the new runner assignments and worker-limit expression. Application tests set controlled filesystem timestamps, verify timestamp changes in scanner and hash-cache cases, and adjust a binary fixture and timer advancement.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to d2408

The runner and test-fixture changes show no concrete merge-blocking behavior. The PR appears mergeable once the reported hosted qualification and exact-head aggregate complete.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 9 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly identifies the primary change: moving PR analysis and Linux unit tests to Blacksmith. It is concise and specific.
Description check Passed The description is detailed and covers the required sections, including the change, rationale, linked issues, testing, review notes, compatibility considerations, and checklist. It explicitly explains…
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔇 Additional comments: 7
config/scripts/ci-dependency-download-cache.test.mjs: 1

298-298: LGTM!

Also applies to: 349-349, 359-370

config/scripts/pr-preflight-gates.test.mjs: 1

19-19: LGTM!

.github/workflows/ci-cache-warmup-platform-jobs.yml: 1

18-18: 🩺 Stability & Availability

The supplied evidence does not show that actionlint reported an unknown runner label. Existing repository workflows already use Blacksmith labels, including blacksmith-4vcpu-ubuntu-2404, but runner availability cannot be established from the provided source.

.github/workflows/unit-tests.yml: 2

178-178: LGTM!


98-106: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

⚠️ Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Patch of /etc/profile.d/blacksmith.sh can silently do nothing.

The Python step replaces an exact string. If the image changes that line, updated == original and the step succeeds without a change. The next step then fails with "Runner login profile overrides fixture HOME". That failure is detectable, so the risk is low. Log a notice when the file is missing or the pattern is not found. This makes the root cause visible.

config/scripts/ci-unit-module-cache.test.mjs: 1

4-4: LGTM!

Also applies to: 150-168

config/oxlint-anti-slop.json: 1

75-80: LGTM!


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2fb6d5b1-774b-46b2-b956-8427e33e3b2e
📥 Commits

Reviewing files that changed from the base of the PR and between a49295f and dffc892.

📒 Files selected for processing: 7
  • .github/workflows/ci-cache-warmup-platform-jobs.yml
  • .github/workflows/pr.yml
  • .github/workflows/unit-tests.yml
  • config/oxlint-anti-slop.json
  • config/scripts/ci-dependency-download-cache.test.mjs
  • config/scripts/ci-unit-module-cache.test.mjs
  • config/scripts/pr-preflight-gates.test.mjs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

@nwparker nwparker changed the title ci: move PR analysis and ARM test shards to Blacksmith ci: move PR analysis and Linux unit tests to Blacksmith Oct 16, 2026
@nwparker
nwparker merged commit 7909e72 into main Oct 16, 2026
31 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants