Skip to content

fix(core): cap the stderr a stdout-only filter forwards on a clean run - #4120

Merged
KuSh merged 1 commit into
rtk-ai:developfrom
KuSh:fix/3772-forwarded-stderr
Sep 20, 2026
Merged

KuSh merged 1 commit into
rtk-ai:developfrom
KuSh:fix/3772-forwarded-stderr

Conversation

@KuSh

@KuSh KuSh commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Fixes one regression from the release review on #3979, introduced by #3772.

Part of a set of six, one per originating PR: #3681, #3552, #3265, #3772 (this), #3857, #3941.

The regression

RunOptions::stdout_only() is documented as "stdout-only to filter, stderr passthrough", and #3772 implemented the passthrough half, which had never existed — before it, a tool reporting on stderr with an empty stdout (golangci-lint with a config or build error) produced no output at all. That part was right and stays.

But it forwarded stderr verbatim on every stdout-only path, so a chatty-but-harmless stderr swamped the filtered stdout. With a stub go emitting 150 go: downloading … lines and one ok … line:

bytes
native go test ./... 3220
through develop 3216 — 0.1 % reduction
the stdout filter alone would give 24

The fix

Stderr still goes through whole whenever it is the report:

  • the command failed (exit_code != 0), or
  • the filter had nothing to show for stdout

Those are the cases the forwarding exists for, and they cover golangci-lint.

Otherwise it is capped — and three things make it give up and forward the lot anyway:

condition why
stderr below the recovery store's own floor not worth a stored file and the eviction that comes with it
no recovery store to point at (RTK_TEE=0, disabled, unwritable) a count of lines nobody can read is worse than the lines; curl_cmd declines the same way
the result would not actually be smaller the note and the hint cost bytes of their own — rtk must emit no more than the command it stands in front of, on either stream

That last one matters: a line-count cap with no byte floor made rtk emit +82 % on an 11-line stderr.

The cap keeps the end, not the head

A tool resolves and downloads before it builds, so its chatter comes first and anything it has to say comes last. A cap on the head kept go: downloading module-1 … module-10 and dropped exactly the two lines worth forwarding:

go: warning: "./..." matched no packages
go: WARNING: module example.com/legacy is deprecated: use example.com/new

Both now survive. The kept lines are sliced out of the original at a byte offset rather than re-joined, so CRLF endings are preserved — str::lines() drops the \r and nothing puts it back.

What the cap holds back goes behind the [full output: …] handle, as src/cmds/README.md requires of any "+N more".

Result

stderr lines native rtk
11 250 B 246 B
20 439 B 435 B
40 859 B 343 B
152 3220 B 354 B — with both warnings intact

Verification

11 unit tests in a new forwarded_stderr_tests module: the tail cap and its note, the failing-run and empty-stdout whole-forward cases, the recovery-floor and no-store and would-not-shrink refusals, CRLF preservation, a non-ASCII stderr sliced without panicking, singular/plural, and last_lines_offset's boundaries (unterminated last line, nothing to skip, empty).

cargo fmt --all, cargo clippy --all-targets and cargo test --all are green, rebased on current develop.

🤖 Generated with Claude Code

Forwarding stderr verbatim keeps a golangci-lint config error from vanishing,
but it also hands back every `go: downloading …` line a cold module cache
produces: a passing `go test` went from 3220 raw bytes to 3216 through rtk,
where the stdout filter alone returns 24.

Stderr still goes through whole whenever it is the report -- the command failed,
or the filter had nothing to show for stdout -- and now also whenever capping
would not pay: below the recovery store's floor, when there is no store to point
at, and whenever the note and the hint would cost more than the lines they
replace. Otherwise the last lines are kept and the rest goes behind the
`[full output: …]` handle, which puts that same `go test` at 354 bytes with its
warnings intact.

The end, not the head: a tool resolves and downloads before it builds, so a cap
on the head keeps the chatter and drops the two lines worth forwarding. The kept
lines are sliced out of the original rather than re-joined, so CRLF endings
survive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@rtk-wshm-sync-bot rtk-wshm-sync-bot Bot added bug Something isn't working core regression labels Sep 19, 2026
@rtk-wshm-sync-bot

Copy link
Copy Markdown

wshm · Automated triage by AI

📊 Automated PR Analysis

🐛 Type bug-fix
🟡 Risk medium

Summary

Fixes a regression in rtk's stdout-only filter mode where stderr was forwarded verbatim on every run, letting chatty-but-harmless stderr (e.g. go: downloading lines) swamp the filtered stdout output. Stderr is now forwarded whole only when it's the actual report (failed run or empty filtered stdout), and otherwise capped to the last N lines with a recovery hint, unless the cap wouldn't actually shrink output or there's no recovery store to point to.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Analyzed automatically by wshm · This is an automated analysis, not a human review.

@pszymkowiak pszymkowiak left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by building the branch and driving src/core/runner.rs with stub go / golangci-lint binaries on PATH (150 go: downloading … lines on stderr, one ok line on stdout), isolated HOME and DB.

Verified

  • develop: rtk go test ./... → 6816 B out of 6820 B native (0.06 % reduction), stderr forwarded verbatim.
  • this PR: 565 B (92 % reduction): ... (+140 earlier stderr lines not shown), the last 10 lines kept, [full output: rtk recall <hash>]; rtk recall <hash> returns all 150 lines. Off-by-one checked (150 = 140 + 10, kept lines are 141–150).
  • Whole-forward cases all hold: exit ≠ 0 (6792 B, no note, exit 1 propagated); empty stdout (golangci config-error shape, 938 B whole, exit 7 propagated); RTK_TEE=0 and RTK_RECALL=0 (whole, zero earlier stderr/full output strings); 11 lines = 486 B below the 500 B floor → whole and nothing stored; 12 lines = 540 B → capped and stored. ANSI in the tail survives.
  • Store side effects only after the floor and line-count checks, matching the curl_cmd convention.

Follow-ups, none blocking

  1. Every clean chatty run now writes a recovery entry (one sqlite row, or one of tee_max_files = 20 in legacy tee mode). 20 clean go test runs in tee mode evict every earlier failure log. Same trade-off curl_cmd/tsc_cmd already make, so not new policy, but the success path could require a minimum saving (a 12-line stderr is stored for an 18 B gain).
  2. Legacy tee mode with tee_on_success writes two files per clean run (<epoch>_go_test.log and <epoch>_go_test-stderr.log) for content the first already holds. Non-default config, bounded.
  3. The CRLF-preservation rationale in the doc comment is unreachable: stream.rs::read_lines_lossy strips \r before the runner sees stderr (verified: 60 CRLF lines in → 0 CR out). The byte-offset slice is still the cheaper implementation; only the comment overstates why.

Approving. CI green on all three OSes.

@KuSh
KuSh merged commit a5dd92e into rtk-ai:develop Sep 20, 2026
12 checks passed
@KuSh
KuSh deleted the fix/3772-forwarded-stderr branch September 20, 2026 18:16
@rtk-release-bot rtk-release-bot Bot mentioned this pull request Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working core regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants