Skip to content

run_streaming's CaptureOnly arm clones the entire child stdout into a field no caller reads #3397

Open

Description

@HetCreep

Two full copies of the child's output are made on the buffered path, one of which appears to be unused entirely.

Provenance: AI-assisted source audit, human-directed, at v0.44.2 (700bdde). Reachability was traced by reading every call site; not measured with a profiler.

1. stream.rs:475 — a clone nobody reads

In the FilterMode::CaptureOnly arm:

filtered = raw_stdout.clone();

Tracing the consumers of StreamResult on that path:

  • src/core/runner.rs:107 reads .raw / .raw_stdout
  • src/cmds/python/uv_cmd.rs:69 likewise

.filtered is only read on the Streaming path (runner.rs:203, search.rs:435), which is a different arm. So for every CaptureOnly invocation the process allocates a second copy of the child's stdout — up to the 10 MiB RAW_CAP — and drops it unused. 34 modules route through runner::run.

2. stream.rs:491 — a third copy, retained alongside the first two

raw = format!("{}{}", raw_stdout, raw_stderr);

StreamResult then holds raw, raw_stdout and raw_stderr simultaneously. With the 10 MiB cap applied to each stream, that is up to ~30 MiB of duplicated capture even on the capped path, against the <5 MB figure in the module docs.

3. runner.rs:14-19 — two more to_string()

let filtered = filtered.to_string();
... never_worse(raw_for_tracking, &filtered).to_string()

Both are unconditional, and never_worse already returns a &str borrow, so the second copy exists only to satisfy the return type.

Suggested directions (untested — no programmer has reviewed these)

  • Leave .filtered empty (or make it Option<String>) in the CaptureOnly arm, since nothing consumes it there.
  • Build raw lazily — a method that concatenates on demand, or have consumers take raw_stdout/raw_stderr directly, so all three are not resident at once.
  • Return Cow<str> from emit_guarded so the common case borrows.

None of these change behaviour; they are allocation removals. Related but separate: #3392 covers rtk grep, where the same copy-everything shape produces a measured 31× memory factor.

Activity

  1. pszymkowiak commented on Aug 4, 2026

    @pszymkowiak
    Collaborator

    wshm · Automated triage by AI

    🔍 Automated Triage

    🐛 Category bug
    🟡 Priority medium
    🎯 Confidence 72%

    Summary

    CaptureOnly path in run_streaming makes redundant/unused clones of child stdout (up to ~30MB vs documented <5MB); trim unused .filtered clone and avoid eager raw concatenation/to_string copies in stream.rs and runner.rs.

    This looks like a simple fix that could be auto-resolved. Repo maintainers can use /wshm fix to attempt it.

    📁 Relevant files
    • src/core/stream.rs
    • src/core/runner.rs
    • src/cmds/python/uv_cmd.rs

    Triaged automatically by wshm · This is an automated analysis, not a human review.

  2. HetCreep commented on Aug 30, 2026

    @HetCreep
    Author

    Refreshing citations. Re-verified at develop @ 0de9610; the defect is unchanged, only my line numbers rotted.

    Provenance: AI-assisted source audit, human-directed, at develop @ 0de9610. Source-read; the reader analysis below was re-traced rather than carried over.

    Strike one sentence

    My report compared the buffer size "against the <5 MB figure in the module docs". There is no such figure at this commit — grep across src/core/ finds only RAW_CAP = 10_485_760 // 10 MiB in stream.rs. Please disregard that comparison.

    Citations, refreshed

    Both primary cites are off by roughly 38 lines, same files. The quoted snippet in the report names raw_for_tracking where the current line reads raw.

    The dead-clone claim, re-traced

    I re-checked this rather than assuming it carried forward. Every non-test reader of .filtered (runner.rs:203, git.rs:1527, search.rs:435) sits on a FilterMode::Streaming call site, while both CaptureOnly sites (runner.rs:107, uv_cmd.rs:69) read only .raw / .raw_stdout / .raw_stderr. The clone in the CaptureOnly arm is genuinely unread work.

    The third sub-claim (two unconditional to_string() calls against a &str-returning function) also still holds, and the incidental "34 modules" figure still checks out.

  3. HetCreep commented on Sep 16, 2026

    @HetCreep
    Author

    Re-checked this against develop @ 5e0f92c (read-only — nothing built or run for this pass). The defect holds exactly as described; only the stream.rs citations need correcting again.

    Citation drift: the two primary stream.rs line numbers — last corrected to 475 and 491 — are now 651 and 667, a ~176-line shift (this pair had already moved ~38 lines once before). Every commit touching stream.rs/runner.rs in the intervening range is unrelated to allocation or copying — d00c514 (let-chains), 7c18567 (edition-2024 migration), 5b1d523/9418aa8/9900d70 (signal-relay handling), 533b964/5be75c2/108a453/c8c2526/f8c2879 (git-blob-show filtering) — so this is pure code growth, not a near-miss fix.

    Current locations, re-verified at 5e0f92c:

    • src/core/stream.rs:651 — filtered = raw_stdout.clone();, inside the FilterMode::CaptureOnly arm (stream.rs:639).
    • src/core/stream.rs:667 — let raw = format!("{}{}", raw_stdout, raw_stderr);. StreamResult (stream.rs:256-262) still carries raw, raw_stdout, raw_stderr, and filtered as four separate, always-populated String fields.
    • Neither CaptureOnly caller reads result.filtered: src/core/runner.rs:95-134 (run_captured_filter) only touches result.raw/result.raw_stdout/result.raw_stderr (lines 115-125) and builds its own filtered text via filter_fn(text_to_filter, exit_code) at line 134; src/cmds/python/uv_cmd.rs:69-76 does the same, building its output via filter_uv_run_output(&result.raw, &result.raw_stdout, &result.raw_stderr, result.exit_code) without ever touching result.filtered.
    • emit_guarded's two to_string() calls are unchanged in substance at src/core/runner.rs:16-24 (line 19 None => filtered.to_string(),; line 21 let shown = crate::core::guard::never_worse(raw, &body).to_string();) — this citation only drifted ~2-5 lines. src/core/guard.rs:17 confirms never_worse still returns a borrow (pub fn never_worse<'a>(raw: &'a str, filtered: &'a str) -> &'a str), so both calls remain purely to satisfy an owned return type nothing downstream requires.
    • A second instance of the same never_worse(...).to_string() pattern now also exists at src/core/runner.rs:147 (let guarded = crate::core::guard::never_worse(raw_for_tracking, &filtered).to_string();), inside run_captured_filter's non-tee-label branch — not in the original report, same shape as the point above.

    No other figures in the original report need correcting — the <5 MB module-docs number struck in an earlier comment on this issue is still absent from stream.rs at HEAD and hasn't resurfaced.

    Tag unchanged: (C) conformance/efficiency — three full copies of the same output plus two no-op allocations, not attacker-reachable on its own, but the same copy-heavy shape underlying the memory report in #3392. Still open.

  4. HetCreep commented on Oct 14, 2026

    @HetCreep
    Author

    Re-checked on v1.0.0 (stable, 2026-10-13): the official release binary plus the v1.0.0 source. Provenance: AI-assisted check, human-directed. "ran" = executed against rtk 1.0.0 with a control; "read" = verified from source only.

    Substance unchanged at v1.0.0 (read; no allocation profile run). Only the locations moved, for a refresh of the cites:

    • src/core/stream.rs:651: filtered = raw_stdout.clone(); in the CaptureOnly arm; :667: let raw = format!("{}{}", raw_stdout, raw_stderr);; StreamResult still returns raw, raw_stdout, raw_stderr and filtered (:684-690).
    • CaptureOnly callers are src/core/runner.rs:300 and src/cmds/python/uv_cmd.rs:70; neither reads .filtered (the .filtered readers at runner.rs:497, git_cmd.rs:2806, search.rs:547 are Streaming sites; only a test at stream.rs:1114 reads it on the capture path).
    • src/core/runner.rs:48,338 still guard_stdout(..).to_string().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions