Skip to content

Record positional argument presence in anonymous telemetry - #634

Open
sdairs wants to merge 1 commit into
fix/608-local-json-error-parityfrom
feat/480-telemetry-positionals
Open

Record positional argument presence in anonymous telemetry#634
sdairs wants to merge 1 commit into
fix/608-local-json-error-parityfrom
feat/480-telemetry-positionals

Conversation

@sdairs

@sdairs sdairs commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

What

Anonymous telemetry recorded command paths and flag names but dropped all positional presence, so materially different invocations produced identical events: local server stop vs local server stop dev, and a bare local use parse failure vs a handler failure after a version was supplied. That is what blocked the August 2026 telemetry investigation (#480).

The payload gains a positionals field carrying the definition-owned ids of the positional slots the user filled — presence only, never a value:

{"command":"local server stop","flags":[],"positionals":["name"],"outcome":"ok", ...}
{"command":"local server stop","flags":["name"],"positionals":[],"outcome":"ok", ...}
{"command":"local use","flags":[],"positionals":[],"outcome":"missing_required", ...}
{"command":"local remove","flags":[],"positionals":["version"],"outcome":"error", ...}

Why it stays anonymous

Every entry is cloned from an Arg definition compiled into the binary — the same "definition-owned, never the user's input" anchoring the existing suggestion field uses — so the field's vocabulary is a closed set that structurally cannot carry argv. Three exclusions keep it honest:

  • only ValueSource::CommandLine slots count, so clap defaults, environment-fed values, and handler-generated names stay absent. That is exactly what makes "the user named it" and "we picked one" distinguishable, so no separate name_source field was needed — the compatibility --name form already lands in flags, so the two naming styles are already distinct.
  • passthrough slots are skipped via a structural marker test (last, trailing_var_arg, allow_hyphen_values) rather than a hand-maintained id list, so argv forwarded to clickhouse-server / clickhouse-client / psql — including everything after -- — is never classified as this CLI's shape, and new passthrough slots inherit the exclusion.
  • on the failed-parse path the walk already stops at -- and never records the unmatched token; it now records only the slot the token reached.

Tests

  • Unit (src/telemetry.rs): bare vs positional vs --name stop; supplied vs missing-required local use/local remove; defaulted positionals excluded; value terminators record nothing; passthrough exclusion on both capture paths; a hostile secret-shaped positional fixture matrix (AWS key, sk-live-…, Postgres URI with password, S3 path, SSH key path, GitHub token) asserted absent from serialized payloads; and a structural guard walking the real command tree that every recorded id is a source-level identifier and every passthrough slot is recognized.
  • End-to-end (tests/telemetry_test.rs): subprocess + wiremock tests pinning the bare/named/flag-named stop distinction, missing-required vs supplied version, and no hostile value on the wire; existing payload-shape assertions extended with positionals.
  • cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test -p clickhousectl (all green), plus cargo check -p clickhousectl --no-default-features --all-targets for the telemetry-compiled-out configuration.

Docs

The first-run notice, the telemetry subcommand's agent help, and the README telemetry section now state the positional privacy boundary and its exclusions (notice/help wording changed from "never values or arguments" to "flag and argument names (never their values)").

Deploy note

The ingest worker's field allowlist needs a matching deploy before the next CLI release, or positionals will be dropped server-side (precedent: #320 / #322, which added outcome/suggestion). Dashboards built on pre-positionals events keep working — the field is additive and absent from older events.

Part of a stacked PR chain: based on fix/608-local-json-error-parity, not main.

Fixes #480

🤖 Generated with Claude Code

@sdairs
sdairs force-pushed the feat/480-telemetry-positionals branch from 92c9f07 to ce0eb74 Compare August 28, 2026 14:46
@sdairs sdairs linked an issue Aug 28, 2026 that may be closed by this pull request
Telemetry recorded command paths and flag names but dropped all
positional presence, so `local server stop` and `local server stop dev`
produced identical events, and a bare `local use` parse failure was
indistinguishable from a handler failure after a version was supplied
(#480).

The payload gains a `positionals` field carrying the definition-owned
ids of the positional slots the user filled — presence only, never a
value. Every entry is cloned from an `Arg` definition compiled into the
binary, so the field's vocabulary is a closed set that cannot carry
argv, exactly like the existing `suggestion` anchoring. Three exclusions
keep it honest:

- only `ValueSource::CommandLine` slots count, so clap defaults,
  environment-fed values, and handler-generated names stay absent —
  which is what makes "the user named it" and "we picked one"
  distinguishable, without a separate source field (the compatibility
  `--name` form already lands in `flags`)
- passthrough slots are skipped via a structural marker test (`last`,
  `trailing_var_arg`, `allow_hyphen_values`), so argv forwarded to
  clickhouse-server/clickhouse-client/psql — including everything after
  `--` — is never classified as this CLI's shape
- on the lossy path the walk already stops at `--` and never records the
  unmatched token; it now records the slot the token reached

Tests: unit coverage for bare vs positional vs `--name` stop, supplied
vs missing-required `local use`/`local remove`, defaulted positionals,
value terminators, passthrough exclusion on both capture paths, a
hostile secret-shaped positional fixture matrix, and a structural guard
that every recorded id in the real command tree is a source identifier
and every passthrough slot is recognized. End-to-end subprocess tests
pin the same distinctions and the no-leak property on the wire.

The first-run notice, `telemetry --help`, and the README telemetry
section now state the positional privacy boundary and its exclusions.

Fixes #480

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sdairs
sdairs force-pushed the feat/480-telemetry-positionals branch from ce0eb74 to f33a195 Compare August 28, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Record positional argument presence in anonymous telemetry

1 participant