[luccinmasirika]

Closes #4390.

Bodies at or under 4KB keep their exact bytes. Above it, JSON is reduced to a compact structural view and other text is windowed to its head. Both paths persist the raw body through the recovery store first and print its hint, so elision is recoverable locally without a re-fetch (an HTTP response, unlike a git object, can never be replayed byte-exactly). Download, header and write-out flags (-o/-O/-i/-I/-w, classified through arg_tokenizer with a curl grammar) and RTK_CURL_RAW=1 force a byte-exact passthrough. The binary passthrough (#1087) is untouched.

Real-world check: rtk curl https://pypi.org/pypi/coverage/json goes from 7,736,373 to 17,519 bytes (-99.8%), recovery hint verified.

20 unit tests, including a >=60% savings assertion on a real fixture. Full suite green.

[rtk-wshm-sync-bot]

wshm · Automated triage by AI

📊 Automated PR Analysis

✨ Type	feature
🟡 Risk	medium

Summary

Reworks rtk's curl output filter to compact large JSON bodies into a structural view and window large text bodies to their head, instead of passing them through unchanged. Download/header/write-out flags and RTK_CURL_RAW=1 still force byte-exact passthrough, and the raw body is persisted to the recovery store with a hint before any elision.

Review Checklist

• Tests present
• Breaking change
• Docs updated

Linked issues: #4390

---

Analyzed automatically by wshm · This is an automated analysis, not a human review.

[luccinmasirika]

Some context on why this reopens the passthrough decision, since there is real history behind it.

The current gate was built in three steps: #1282 reported silent corruption when stdout is piped and suggested the isatty passthrough, #1536 reported the mid-string truncation that produced invalid JSON (... (N bytes total) inserted inside a string value, breaking rtk curl | jq), and #1602 fixed both with the JSON passthrough plus the IsTerminal gate. #1587 tried a middle ground (a valid JSON envelope describing the truncation) and was closed. Given the tools available at the time, full passthrough was the right call: the old truncation was byte-blind, could cut inside a JSON token, and saved nothing worth the corruption.

Two things have changed since.

First, the hook now has PipelineSafety::None for curl (src/discover/rules.rs), so a piped curl ... | jq is never rewritten to rtk curl. The downstream parser that #1282 and #1536 were protecting no longer reaches this filter in hook usage. The consumer of rtk curl output is the agent reading through the harness pipe.

Second, this PR does not reintroduce the #1536 cut. The compact view parses the body first, so it never truncates mid-token; every elision is marked, the banner on the first line says what happened, and the raw body goes to the recovery store before anything is elided (no store, no filtering). If parsing fails, the JSON branch does nothing to the body. Worth noting: the #1536 repro was 4,048 bytes, which stays under the 4KB threshold here, so that exact case still passes through byte-identical.

The one contract change is for someone manually typing rtk curl <url> | jq on a body over 4KB: they now get the compact view instead of valid JSON. That path has three outs: the banner makes it obvious, RTK_CURL_RAW=1 disables it, and rtk proxy curl has always been the bypass. Happy to gate it differently if you prefer another default.