Skip to content

fix(tracking): keep the user's arguments out of the telemetry command label - #4123

Merged
KuSh merged 1 commit into
rtk-ai:developfrom
KuSh:fix/1298-telemetry-command-label
Sep 23, 2026
Merged

KuSh merged 1 commit into
rtk-ai:developfrom
KuSh:fix/1298-telemetry-command-label

Conversation

@KuSh

@KuSh KuSh commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

From the release review on #3979, which flagged low_savings_commands under #1298. Independent of the six regression PRs — it shares no file with any of them and branches off develop directly.

The leak

The telemetry label was the first three words of rtk_cmd. That field is RTK's own, but most of what it holds is the user's command line, so the third word is routinely an operand:

rtk ls /usr/bin                           445 rows
rtk curl https://mockhttp.org/robots.txt   26
rtk tree src/                              20
rtk ls /tmp/claude-1000/-home-user-…/scratchpad/…
rtk wc src/main.rs
rtk ls --all=x

The TOML-filter path is worse: it stores the user's entire invocation after an rtk:toml prefix, so the label could carry a jq program and the file path it ran against.

Measured over a real history database — 329,583 rows, 26,934 distinct rtk_cmd — 201 of 2606 labels carried a path, a URL, a flag value or a search term.

top_passthrough had exactly this leak and was fixed in 15f666dd by grouping on the tool alone. low_savings_commands was left behind.

Why "just take fewer words" doesn't work

Grouping on the first word collapses everything, since every label begins with rtk. Taking two words loses the granularity that makes the figure worth sending — rtk git log and rtk git status are different filters with different savings.

Taking three words when the first is rtk and two otherwise only moves 201 → 195: it fixes the rtk:toml family but not the main leak, because for a single-level command the third word is the operand.

The distinction is not word count. It is whether the word is RTK's routing or the user's data.

What it does now

Keep the prefix, the tool as a basename, and a third word only when the tool routes by a subcommand of its own and that word is shaped like a subcommand (^[a-z][A-Za-z0-9:_-]*$).

rule distinct labels carrying an argument
take(3) (before) 2606 201
first word rtk ? 3 : 2 2589 195
routing-aware 2054 3
+ basename on the tool 2054 0

The basename step clears the last three (rtk ./gradlew, rtk ./mvnw, rtk:toml /tmp/rtk-pr) and merges ./gradlew with gradlew, so the labels improve rather than degrade. Granularity is intact: rtk git log, rtk git status, rtk git show, rtk gh pr all stay distinct.

The router list

A superset of the tools that declare subcommands in discover::rules, which is the authority for the ones it covers — command_label_matches_the_discover_rules fails if a tool gains subcommands there and is not added here. The rest are listed because that table does not describe them: it carries no subcommands for git, the most routed tool there is.

Cross-checked by a test rather than read at runtime, because nothing in core depends on discover (the dependency runs the other way, 8 references) and this is not worth inverting it for.

That test paid for itself three times on its first runs, each a real defect in this change:

  • sbt testOnly — the subcommand shape rejected camelCase
  • phpstan — declares subcommands, was missing from the list
  • bun pm ls — a two-word subcommand, which a three-word label cannot carry; it keeps pm, coarser than the table and never an argument

A tool missing from the list keeps two words. That costs granularity and never leaks — the direction this has to fail in.

Not changed

The review also filed "telemetry query now admits negative-savings rows" under the same PR. That is deliberate and documented in the function's own docstring — "A net-regressing command (negative rate) is listed: it is the filter most in need of improvement" — added by the same commit that changed the predicate. It is also reachable, which the review could not establish: rtk git worktree sits at −7.7 % over 2103 calls in a real database. Left as-is.

Verification

6 unit tests: the real leaking shapes, the routed subcommands that must survive, an operand in the subcommand slot being dropped anyway (HEAD~3..HEAD, --git-dir=…, a quoted path, a URL), degenerate input, the discover::rules cross-check, and a property test asserting no label contains /, =, ", ', @, ~ or ...

cargo fmt --all, cargo clippy --all-targets and cargo test --all are green on current develop.

🤖 Generated with Claude Code

@rtk-wshm-sync-bot

Copy link
Copy Markdown

wshm · Automated triage by AI

📊 Automated PR Analysis

🐛 Type bug-fix
🟢 Risk low

Summary

Fixes a telemetry leak where the low_savings_commands label included the user's arguments (paths, URLs, flags) by naively taking the first three whitespace-separated words of the RTK command. The new command_label function keeps only the RTK prefix, the tool as a basename, and a third word solely when the tool is a known subcommand-router and that word is shaped like a subcommand.

Review Checklist

  • Tests present
  • Breaking change
  • Docs updated

Linked issues: #1298


Analyzed automatically by wshm · This is an automated analysis, not a human review.

@KuSh KuSh mentioned this pull request Sep 21, 2026
Comment thread src/core/tracking.rs Outdated
@KuSh
KuSh force-pushed the fix/1298-telemetry-command-label branch from d5d3b4b to 5fcd048 Compare September 23, 2026 19:57
… label

`low_savings_commands` labelled each command with the first three words of
`rtk_cmd`. That field is RTK's own, but most of what it holds is the user's
command line, so the third word is routinely an operand: `rtk ls /usr/bin`,
`rtk curl https://…`, `rtk grep <pattern>`, and the TOML filter path stores the
user's whole invocation after its `rtk:toml` prefix. Over a real history
database, 201 of 2606 labels carried a path, a URL, a flag value or a search
term.

`top_passthrough` had the same leak and was fixed by grouping on the tool alone.
That is too coarse here, because every label begins with `rtk` and the figure is
only worth sending per filter -- `rtk git log` and `rtk git status` have to stay
apart.

So the label keeps the words RTK chose: the prefix, the tool as a basename, and
a third word only for a tool that routes by a subcommand of its own, and only
when it is shaped like one. Over the same database that is 0 labels carrying an
argument, with the per-subcommand granularity intact.

The router list is a superset of the tools that declare subcommands in
`discover::rules`, cross-checked by a test rather than read at runtime, since
nothing in `core` depends on `discover`. A tool missing from it keeps two words,
which loses granularity and never leaks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@KuSh
KuSh force-pushed the fix/1298-telemetry-command-label branch from 5fcd048 to e58ae41 Compare September 23, 2026 20:01
@KuSh
KuSh merged commit 005eb9c into rtk-ai:develop Sep 23, 2026
11 checks passed
@KuSh
KuSh deleted the fix/1298-telemetry-command-label branch September 23, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants