Skip to content

test(miner): add regression coverage for the two-tier stalled-output timeout - #5575

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nickmopen:feat/stall-timeout-regression
Jul 13, 2026
Merged

test(miner): add regression coverage for the two-tier stalled-output timeout#5575
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
nickmopen:feat/stall-timeout-regression

Conversation

@nickmopen

Copy link
Copy Markdown
Contributor

Summary

Adds the missing regression coverage (#5196) for the two-tier stalled-output timeout on the CLI-subprocess coding-agent driver (packages/gittensory-engine/src/miner/cli-subprocess-driver.ts). The firstOutputTimeoutMs fast-fail path — added after the #4994/#5053 production outage where a zero-output claude/codex process was only killed at the full timeoutMs and masked as a generic timeout — had no dedicated test of its own.

Three cases, using the driver's existing injected-spawn fake, named to map to the #4994/#5053 outage:

  1. Stall — zero stdout past firstOutputTimeoutMs ⇒ asserts the distinct claude_stalled_no_output error + summary, and that it is not conflated with the generic _timeout_ error.
  2. Slow but producing — output before the first-output deadline ⇒ asserts it is not killed early and completes normally.
  3. Full-timeout regression guard — output produced, then no exit before the full timeoutMs ⇒ asserts the pre-existing claude_timeout_5000ms path is unchanged (the new fast-fail didn't alter the old behavior).

Test-only, layered on the already-merged timeout feature; touches only the driver's own test file — no driver/attempt/governor control-flow changes.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused (one test file) and does not mix unrelated changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves — Closes #5196.

Validation

  • git diff --check
  • npx vitest run test/unit/cli-subprocess-driver.test.ts — 42 pass (existing + the 3 new cases)
  • typecheck clean on the changed file
  • Rebased onto latest main (clean); diff is exactly one file

If any required check was skipped, explain why:

  • Test-only change (test/**). No src/**/apps/**/package changes, so Codecov (which ignores test/**) and the UI/MCP jobs are path-skipped; there is no product code to cover.

Safety

  • No secrets, wallets, hotkeys, coldkeys, PATs, private keys, raw trust scores, private rankings, or maintainer evidence are exposed — the fake spawn returns only synthetic stdout.
  • Public GitHub text stays sanitized and low-noise.
  • No changelog edited.

Notes

  • Asserts the distinctness invariant the feature exists to provide (stall vs full timeout must be separately countable in logs/Sentry), not just the happy path.

Closes #5196

…timeout

Covers the fast-fail first-output deadline on the CLI-subprocess coding-agent driver (the
firstOutputTimeoutMs path, JSONbored#4994/JSONbored#5053), which had no dedicated regression test of its own:

- zero stdout past firstOutputTimeoutMs -> distinct `_stalled_no_output` error, not a full timeout
- output before the first-output deadline -> not killed, completes normally
- output then no exit before the full timeoutMs -> the pre-existing `_timeout_` path is unchanged

Test-only, layered on the already-merged timeout feature; touches only the driver's own test file.

Closes JSONbored#5196
@nickmopen
nickmopen requested a review from JSONbored as a code owner July 13, 2026 06:39
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.98%. Comparing base (aa7382d) to head (a884c6a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5575   +/-   ##
=======================================
  Coverage   94.98%   94.98%           
=======================================
  Files         573      573           
  Lines       45469    45469           
  Branches    14680    14680           
=======================================
  Hits        43187    43187           
  Misses       1531     1531           
  Partials      751      751           
Flag Coverage Δ
shard-1 44.03% <ø> (-0.01%) ⬇️
shard-2 35.68% <ø> (+0.01%) ⬆️
shard-3 32.03% <ø> (-0.09%) ⬇️
shard-4 31.03% <ø> (-1.85%) ⬇️
shard-5 32.78% <ø> (+0.90%) ⬆️
shard-6 44.56% <ø> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 13, 2026
@loopover-orb

loopover-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-13 06:46:38 UTC

1 file · 2 AI reviewers · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This is a test-only PR that adds three regression tests to the CLI-subprocess driver's test suite, covering the two-tier stalled-output timeout (fast-fail on zero stdout vs. slow-but-producing vs. full-timeout regression guard). The tests use the driver's existing injected-spawn fake pattern and CI already confirms all 42 tests pass, including the 3 new ones. The PR is narrowly scoped to one test file, closes the linked issue #5196, and the assertions (distinct `claude_stalled_no_output` error, non-early-kill on partial output, unchanged `claude_timeout_5000ms` path) map directly to the documented `firstOutputTimeoutMs` behavior visible in the driver source excerpt.

Nits — 5 non-blocking
  • The `fakeSpawn` helper's `stalledNoOutput` option isn't shown in this diff/context, so it's worth double-checking it correctly simulates zero-stdout + kill-at-firstOutputTimeoutMs semantics rather than just setting a flag without exercising real driver logic.
  • The inline comments restate what the assertions already show (e.g. 'a real spawn clears the first-output timer on that byte') — fine per this repo's convention but slightly verbose for test-only code.
  • Consider also asserting `result.transcript` or `result.errorDetail` content in the stalled case to guard against future regressions in what gets surfaced alongside the error code.
  • If not already covered elsewhere, a fourth case exercising `codex` (not just `claude`) through the same two-tier timeout would strengthen the regression guard given the driver supports both commands.
  • nit: `test/unit/cli-subprocess-driver.test.ts:487` says a real spawn clears the first-output timer, but this fake only returns a prebuilt successful result, so the test proves driver success handling after stdout rather than the timer-clearing behavior itself.
Signal Result Evidence
Code review ✅ No blockers 2 reviewers, synthesized
Linked issue ✅ Linked #5196
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 140 registered-repo PR(s), 88 merged, 12 issue(s).
Contributor context ✅ Confirmed Gittensor contributor nickmopen; Gittensor profile; 140 PR(s), 12 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ⚠️ ℹ️ Insufficient signal risk: clean · value: insufficient-signal — Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed). LLM value judgment: moderate — The PR fills a concrete, previously-missing regression-test gap for a timeout code path directly tied to a documented prior production outage (#4994/#5053), which is exactly the kind of narrow, high-signal test coverage this review guide prioritizes.
Linked issue satisfaction

Addressed
The PR adds exactly the three test cases the issue calls for (zero-stdout stall with distinct error, slow-but-live output not killed early, and a full-timeout regression guard), reuses the existing injected-spawn fake, and names/comments the cases with clear #4994/#5053 cross-references, matching the issue's stated scope and test-only constraint.

Review context
  • Author: nickmopen
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 140 PR(s), 12 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 246e7ec into JSONbored:main Jul 13, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a regression test for the two-tier stalled-output timeout in the CLI-subprocess driver

1 participant