hook-telemetry: envelope schema validates nothing while ten suites hand-transcribe the field list (deepening candidate)

[kyle-sexton]

Context

docs/conventions/hook-telemetry/envelope.schema.json is a published interface with no adapter: a repo-wide grep across *.sh / *.mjs / *.py / *.json finds no consumer beyond its own $id line (reproduced 2026-08-27). Nothing executes it.

Meanwhile the literal line for field in schema_version timestamp hook hook_event status duration_ms data; do appears in 10 test suites (go-format:351, ruff-format:380, biome-format:351, bash-format:508, powershell-format:930, typos-format:1586, markdown-format:1886 and 1933, actionlint:301, eol-normalizer:213), each followed by the same four-line jq -e "has(...)" body. Each copy is a hand transcription of the schema, and a transcription can go vacuous without any gate noticing: #3394 (d6e7830) is the concrete failure, where go-format's stdout-leak assertion grepped a variable the telemetry run never assigned and "passed unconditionally ... leaving the stdout-parity contract unguarded while appearing tested."

Deletion test: concentrates. Two adapters are natural: producers emit envelopes in production, and one schema-driven check consumes them in tests.

Proposed work

• One envelope-check helper driven by the actual schema file (the pinned check-jsonschema is already in the toolchain), consumed by the suites in place of the transcribed loops.
• Make the check provably non-vacuous: a deliberately malformed envelope must fail it (asserted in the helper's own suite).
• Keep the per-suite stdout-parity assertion (envelope never on the hook's own stdout), pointed at real captured output.

Acceptance criteria

• envelope.schema.json is executed by at least one code path (the shared check), with the schema file as the single source of the field list.
• The ten transcription loops are replaced by calls to the shared check.
• A malformed envelope demonstrably fails the check (mutation-style assertion in the helper's suite).

References

• docs/conventions/hook-telemetry/envelope.schema.json, docs/conventions/hook-telemetry/README.md
• The ten suites listed above; fix(go-format): capture the telemetry run's own stdout in the leak assertion #3394 / commit d6e7830 (the vacuous-assertion failure mode)
• Constraint to weigh: docs/conventions/shell-test-helpers/README.md governs where shared test code may live; this proposal should be reconciled with it (repo-tooling layer vs per-plugin suites).
• Parent candidate: the shared formatter-hook prologue (filed separately, top recommendation).
• Found by the /architecture:improve deepening scan (2026-08-27); shallow-signal reproduced before filing. Dependency category: local-substitutable. Recommendation: Strong.

[kyle-sexton]

This was generated by AI during triage.

Claimed for triage by autonomous lane instance vm-triage-202609050304. Lease expires 2026-09-06T20:15:00Z. Triage claim only, not an execution claim.

---

Generated by Claude Code

[kyle-sexton]

This was generated by AI during triage.

Triage verdict

Category: Task (confirmed, and slightly worse than filed). State: verified → blocked by #3408 decision. Work class: structural. Priority: medium.

Blocked by #3408, recorded as a native dependency edge. #3408 is the decision carrier for this cluster; one human touch there unblocks this.

Re-verification against current main (2026-09-06)

Both claims hold, and the second has grown:

• The schema still validates nothing. A repo-wide search for references to the envelope schema file finds it named only in its own definition and in the convention README beside it. No script, no suite, no check executes it. It remains a published interface with no adapter, exactly as filed.
• The hand-transcribed field list has spread from ten copies to eleven. The same literal field-list loop now appears in the ten suites originally named plus the shared library's own test suite. Ten days, one more copy. That is the strongest available evidence that transcription is the path of least resistance here and will keep being taken until the shared check exists.

The failure mode cited in the body (a transcribed assertion going vacuous while still appearing to pass) is not hypothetical; the referenced commit is a worked instance of it.

Why blocked rather than agent-ready

The work itself is well-specified, but where the shared check may live is the same question #3408 carries. These suites sit in ten different plugins, so a helper they all consume is cross-plugin shared code, which the vendoring ADR governs, while the shell-test-helpers convention separately governs where shared test code may live and draws a line between the repo-tooling layer and per-plugin suites. The body flags this reconciliation as needed and does not resolve it.

Answering it independently here would risk contradicting whatever #3408 decides for the production-side prologue, and having the test layer and the production layer land on different answers is worse than either answer.

What is ready the moment #3408 resolves

The scope is clear and the acceptance criteria in the body are sound. Two points worth carrying forward:

• The non-vacuity assertion is the important one. A shared check that never fails is exactly the failure this issue exists to prevent, so the helper's own suite must prove a malformed envelope fails it. Without that, the change substitutes one unverified assertion for eleven.
• The stdout-parity assertion must stay per-suite and stay pointed at real captured output. It is a different property from schema conformance (the envelope must not appear on the hook's own stdout), and folding it into the shared check would lose it.

One addition to the body's criteria: the schema file should gain a consumer test as well as a consumer, so that a future change deleting the last caller fails rather than silently returning the schema to its current orphaned state.

Triage claim released by vm-triage-202609050304.

---

Generated by Claude Code