Skip to content

Restore main unit checks for plugin routing and prompt receipts - #37345

Merged
nwparker merged 3 commits into
mainfrom
brennanb2025/lt1031-main-ci-tests-2
Nov 2, 2026
Merged

nwparker merged 3 commits into
mainfrom
brennanb2025/lt1031-main-ci-tests-2

Conversation

@brennanb2025

@brennanb2025 brennanb2025 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor
Files Added Deleted Net
Test 2 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​55 $\color{#cf222e}{\Huge{\mathbf{−}}}$​31 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​24
Prod 1 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​24 $\color{#cf222e}{\Huge{\mathbf{−}}}$​5 $\color{#1a7f37}{\Huge{\mathbf{+}}}$​19

Main's unit shards 1/12 and 12/12 fail independently of PR changes. The plugin prompt-hook change in 60d262ebee9 (#37295) added an async facade, so the default prompt receipt now takes six promise reactions instead of the test's hard-coded five. It also pushed plugin sources using the routing fallback beyond the existing limit. The glyph resolver introduced in 7df246b9d1a (#36814) had never received a specific route. The editor screenshot spec added in f908d2f40ab (#37302) subsequently landed without a source/helper route, causing two more assertions in that same routing suite to fail.

This updates CI coverage without changing shipped prompt delivery. Glyph changes select the declarative-panel journey. Screenshot replay, its scenario loader and the scenario's panel checker and test loader select the panel screenshot journey only. The editor screenshot's command-line code selects the editor screenshot journey, and shared capture code (including the status-bar shot and default output folder) selects both. Before, several of those command-line files selected no end-to-end test at all. The prompt test now holds the write, proves the receipt remains pending, then releases it and verifies acceptance without advancing fake timers. That protects completion ordering and the absence of a timer delay while allowing promise composition to change.

Plan: repair the existing source routes and replace the implementation-specific tick count with the observable receipt contract. The fix-broken-main task explicitly skips the external-reference and plan-review workflows. No capability, default, saved data, paired-client behavior or UI choice changes, and no prior ruling is reversed.

Light-path classification:

  • Small: 9 changed CI-routing lines in one non-test file; all remaining changes are in two existing test files.
  • Outside risky areas: no shipped runtime, persistence, ordering, wire, process, auth, orchestration, release, build or packaging code changes; the prompt assertion exercises existing behavior only.
  • Not a repeat: the touched-file history shows feature route additions and the original prompt test, with no prior fix/revert/incident for the missing glyph/editor route or hard-coded receipt-count failure.
  • Local effect: CI routing and tests only. Existing sibling routing is extended; there is no separate routing mechanism or coverage relaxation.

Validation: both named suites reproduced on unchanged main 8d1453fba98 (4 failures, 34 passes), and pass after the changes (42 tests). Ran with Node through tc-gate using config/vitest.config.ts. The task restricts local validation to these named suites; lint/typecheck remain CI gates. No app launch is needed for this test/CI-only change.

CI: the requested fail-fast watch on initial head 298254883b0 stopped at the unchanged-main TS2307 failure in src/main/native-chat/agent-model-catalog/agent-model-catalog-scaling.test.ts:11 (missing ../../../shared/agent-session-conversation-view), last touched by #37191. Job: https://github.com/stablyai/orca/actions/runs/41397253386/job/125121215960. Unit shards were pending at that return. The final test-only follow-up df134cc24e9 strengthens the held-write assertion by draining promise work at time zero; both named suites passed again (42/42). No second CI watch was started under this task's one-watch limit.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 2 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1f71aba9-de0d-4fdb-bbdc-acf341b341a7
Commits

Reviewing files that changed from the base of the PR and between e30202e and 2560793.

Files selected for processing: 2
  • config/scripts/pr-plugin-e2e-routing.test.mjs
  • config/scripts/pr-plugin-e2e-source-routes.mjs
Walkthrough

The plugin E2E source routes and routing tests now cover additional icon, view editor, replay, and test registration sources. The prompt receipt test now verifies that a deferred write keeps the receipt unsettled until the write completes, then checks the resolved handle and accepted status.

Merge Risk: Low · previous review df134

Current prompt delivery is unaffected, but the test can miss a timer-dependent regression. Strengthen the check before relying on it.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title names the main change: restoring main's unit checks for plugin routing and prompt receipts. The raw summary shows CI routing additions in pr-plugin-e2e-source-routes.mjs and pr-plugin-e2e-ro…
Description check Passed The description follows the repository template in substance. It explains the change and its reasons, and it cites the prompt-hook and routing causes. It describes the test change, validation runs, an…
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: restoring main's unit checks for plugin routing and prompt receipts. The raw summary shows CI routing additions in pr-plugin-e2e-source-routes.mjs and pr-plugin-e2e-routing.test.mjs, plus a rewritten prompt receipt test in agent-prompt-default-path.test.ts. The title is specific and matches the changeset, so it passes.

Full details: Description check

Explanation

The description follows the repository template in substance. It explains the change and its reasons, and it cites the prompt-hook and routing causes. It describes the test change, validation runs, and the scope limit to CI routing and tests. It gives a risk assessment, low-risk classification reasoning, and a CI status note. Several template sections are missing or incomplete: the ELI5 section is absent, the Linked Issue section has no issue (only a 'Fixes #' placeholder, and the author does not state an issue number), Visual Proof is not marked N/A with a reason, the Risk section has no explicit 'Level' value and no explicit backout procedure, the Testing checkboxes are not ticked, and the Review, Agent skill upstream boundary, and Checklist sections are not addressed. The description is mostly complete, so the check passes, but the author should fill in the missing sections.

Resolution

Add an ELI5 section in plain language. Replace the empty 'Fixes #' line with a linked issue, or explain why none applies. Write N/A with a reason under Visual Proof. Set the Risk level to Low, Medium, or High, and state how to revert the change. Mark the Testing checkboxes that apply, and complete the agent-skill boundary checkbox and the Checklist.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 3 files.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick comments: 1
src/main/runtime/agent-prompt-default-path.test.ts: 1

70-70: Trivial · Functional Correctness · Quick win

Drain the receipt promise chain without advancing fake timers.

One await Promise.resolve() is not enough to exercise the receipt facade and the settled callback. Drain the five promise-job turns used by the previous receipt check. This detects both an accepted receipt returned before the deferred write completes and a receipt that incorrectly waits for a zero-delay timer.

Suggested fix
-      await vi.advanceTimersByTimeAsync(0)
+      for (let tick = 0; tick < 5; tick += 1) {
+        await Promise.resolve()
+      }

Review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 64ae0cf1-bdb7-4a8d-81e5-d02e5f7df2b0
Commits

Reviewing files that changed from the base of the PR and between 999160b and df134cc.

Files selected for processing: 3
  • config/scripts/pr-plugin-e2e-routing.test.mjs
  • config/scripts/pr-plugin-e2e-source-routes.mjs
  • src/main/runtime/agent-prompt-default-path.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

@nwparker
nwparker force-pushed the brennanb2025/lt1031-main-ci-tests-2 branch from df134cc to e30202e Compare November 2, 2026 06:16
@nwparker
nwparker merged commit d712485 into main Nov 2, 2026
42 checks passed
@nwparker
nwparker deleted the brennanb2025/lt1031-main-ci-tests-2 branch November 2, 2026 06:34
@brennanb2025

Copy link
Copy Markdown
Contributor Author

Review status: done on 2560793 (merged before the final design ruling)

The problem. Main's own unit checks were red, so every open PR's CI failed regardless of its changes. Two tests broke: (1) the plugin end-to-end routing test, because recently added plugin files (the icon glyph resolver, the editor screenshot journey and its helpers, the screenshot replay and scenario loader) had no specific route to the end-to-end test that exercises them, pushing the shared "unrouted" count over its 10% limit; (2) a prompt-delivery test that pinned an exact number of promise steps, which changed when plugin input hooks put sending behind an async layer.

What this PR changes for users. Nothing shipped. CI routing and tests only.

What was fixed during review.

  • The editor screenshot's command-line code (editor shot, status-bar shot, default output folder) and the scenario run's panel checker and test loader now route to the end-to-end journeys that actually run them. Before, a change to those files selected no end-to-end test at all.
  • The editor screenshot spec is no longer attached to every screenshot source; replay and scenario-only files select only the panel screenshot journey. Every other file's selection is unchanged (compared across all tracked files).
  • The prompt test now holds the write, checks the receipt is still pending, releases it, and checks it is accepted without any timer running. That keeps the original protection (acceptance only after the write, no added delay) without pinning a step count. A mutation run confirmed it fails on early acceptance and on a timer before or after the write.

Verified. All required CI checks pass on 2560793. The two changed test files pass, 53/53. No app run was needed, since only tests and CI routing changed.

Deferred to a follow-up (design ruling, all three panel seats agreed on the design). The routing test only limits the share of unrouted plugin files (10%), so a new plugin file can still land with no targeted end-to-end test, and plugin command-line files aren't counted at all. On this head 49 of 511 files are unrouted against a limit of 51.1, so main goes red again after about three more. The ruling: in a separate PR, replace the ratio with a per-file rule: every plugin source, including command-line sources, must match a specific route or appear by name in an exception list. The list starts with today's 49 unrouted files plus 31 command-line files with no route, and must match the real gaps exactly so stale entries can't hide new ones. The follow-up adds no new workflow, does not route every plugin change to every plugin test, and changes no selections. This PR merged before that ruling, so nothing in it changes.

Noticed, not in scope. Several plugin PRs merged before their own unit runs finished, and those runs later failed. That is how this breakage reached main.

For Brennan's review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants