fix(bench): pin the curl fixtures instead of fetching a random payload

The curl and wget cases fetched https://mockhttp.org/json/1, which returns a
randomly chosen body per request:

  call 1: 152 bytes  {"metrics":...
  call 2: 244 bytes  {"config":...
  call 3: 102 bytes  {"message":...

bench() calls the raw command and the rtk command separately, so the two sides
received different payloads and the ratio between them was noise. Whether the
build passed was a coin flip on which pair it drew — it passed on develop at
3cb7107 and failed twice on the very next PR, which touched no filter code.

That matters more now that CD's ci-gate blocks releases on CI: a third party's
mock API sat in the critical path of every release.

Points all three cases at pinned file:// fixtures. curl and the filter stay in
the loop; only the nondeterminism goes. Note rtk passes JSON through unchanged,
so these cases will now sit honestly at ~0% savings rather than swinging between
a fake win and a build failure.

There is no filter defect here: core::guard::never_worse already prevents rtk
emitting more tokens than raw, and curl routes through it via emit_guarded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: kylehgc

.github/README.md

@@ -16,8 +16,15 @@ When a coding agent runs `cargo test`, `git log`, `npm install`, or `pytest`, th
 output goes straight into its context window — thousands of lines of progress bars,
 timestamps, and repetition, most of which the model cannot use. rtk sits in front of
 those commands, runs them for real, and passes back a filtered version that keeps the
-errors, the failures, and the diagnostics while dropping the noise. Typical reduction
-is 60–90% of bash output.
+errors, the failures, and the diagnostics while dropping the noise. Across rtk's own
+benchmark suite — 73 cases, run with [`scripts/benchmark.sh`](../scripts/benchmark.sh) —
+aggregate output came to **541,111 → 123,205 tokens, a 77% reduction**. Individual
+commands vary enormously: some are cut by 90%, and some are passed through untouched
+because filtering them would lose information or gain nothing.
+
+That is a measure of **bash output**, not of your bill, and rtk ships no tokenizer —
+it estimates tokens as bytes/4, so the ratios are sound and the absolute counts are
+approximate.
 
 It is a transparent proxy: `rtk cargo test` runs `cargo test`, exits with the same
 status code, and works for commands it has no specific filter for. A hook can rewrite

scripts/benchmark.sh

@@ -345,17 +345,27 @@ bench "wc" "wc Cargo.toml src/main.rs" "$RTK wc Cargo.toml src/main.rs"
 # curl
 # ===================
 section "curl"
+# Pinned local fixtures, not a live host. These cases used to fetch
+# https://mockhttp.org/json/1, which returns a *randomly chosen* payload per
+# request — so the raw call and the rtk call received different bodies and the
+# ratio between them was noise. It failed the build at random, and since CD's
+# ci-gate blocks releases on CI, a third party's mock API could stop all
+# releases. file:// keeps curl and the filter in the loop while making the
+# comparison deterministic and offline.
+CURL_JSON_URL="file://$(pwd)/tests/fixtures/benchmark_curl.json"
+CURL_TEXT_URL="file://$(pwd)/tests/fixtures/benchmark_curl.txt"
 if command -v curl &> /dev/null; then
-  bench "curl json" "curl -s https://mockhttp.org/json/1" "$RTK curl https://mockhttp.org/json/1"
-  bench "curl text" "curl -s https://mockhttp.org/robots.txt" "$RTK curl https://mockhttp.org/robots.txt"
+  bench "curl json" "curl -s $CURL_JSON_URL" "$RTK curl $CURL_JSON_URL"
+  bench "curl text" "curl -s $CURL_TEXT_URL" "$RTK curl $CURL_TEXT_URL"
 fi
 
 # ===================
 # wget
 # ===================
 if command -v wget &> /dev/null; then
   section "wget"
-  bench "wget" "wget -qO- https://mockhttp.org/json/1" "$RTK wget https://mockhttp.org/json/1"
+  # Same pinned fixture, same reason as the curl cases above.
+  bench "wget" "wget -qO- $CURL_JSON_URL" "$RTK wget $CURL_JSON_URL"
   rm -f 1 2>/dev/null
 fi
 

tests/fixtures/benchmark_curl.json

@@ -0,0 +1 @@
+{"metrics":{"cpu":45.2,"memory":78.5,"disk":62.3,"network":{"inbound":"1.2MB/s","outbound":"0.8MB/s"},"uptime":"15 days","services":[{"name":"api","status":"healthy"},{"name":"database","status":"healthy"},{"name":"cache","status":"warning"}]},"config":{"theme":"dark","language":"en","notifications":{"email":true,"push":false,"sms":false},"privacy":{"profile":"public","activity":"friends"}},"message":"Hello from a pinned fixture","timestamp":"2026-08-01T00:00:00Z"}

tests/fixtures/benchmark_curl.txt

@@ -0,0 +1,7 @@
+User-agent: *
+Disallow: /admin
+Disallow: /private
+Disallow: /internal
+Allow: /
+Crawl-delay: 10
+Sitemap: https://example.invalid/sitemap.xml