Repository navigation
docs(adr): record the #1280 soak verdict, ADOPT TEMPO / mctl-native only - #2071
Conversation
The owner closed the soak on 2026-10-22, 30 h 59 min before the declared end, for operator capacity. The window amendment records that change to a declared parameter openly, and the Decision opens with it, so the verdict is not read as resting on the full declared window. Every Tempo cell is scored from a measurement cited in the rubric. The other candidates' cells are null with the scope-amendment reason, as the rubric requires for anything not measured. The freeze test now enforces the scored contract once a verdict is set, instead of failing on it. The record also keeps findings that do not flatter the result: a bounded trace-by-id returned 5,092 of 6,930 in-window spans (unexplained; the ingest counters show no loss), and the verdict's own reads pushed the pod past the 80% memory line after the cutoff. Refs #1280
|
Claude finished @mashkovd's task in 5m 4s —— View job Review complete ✅
Verdict: APPROVED — 0 P1/P2, 1 P3. What I checked beyond reading the prose:
One inline P3 nit posted on No P1/P2 findings — the math, the amendment caveat, and the test's dual-mode contract all hold together. |
Antigravity (agy) review — informationalModel: gemini-3.8-flash-high · commit No significant issues found. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| langfuse: {score: null, citation: ""} | ||
| phoenix: {score: null, citation: ""} | ||
| signoz: {score: null, citation: ""} | ||
| traceway: {score: null, citation: "not deployed in this round: the 2026-10-10 bake-off scope amendment limited #1280 to Tempo, so this cell was not measured"} |
There was a problem hiding this comment.
P3: The citation string "not deployed in this round: the 2026-10-10 bake-off scope amendment limited #1280 to Tempo, so this cell was not measured" is repeated verbatim 24 times (4 non-Tempo candidates × 6 dimensions). Not a correctness issue — the test only requires a non-empty citation per cell — but a YAML anchor (e.g. ¬_deployed) would remove the duplication if this file gets touched again.
There was a problem hiding this comment.
Deferred (P3, cosmetic). The repeated citation is deliberate for now: each cell stays self-describing when read alone. An anchor can come with the next edit to this file.
There was a problem hiding this comment.
No P1/P2 findings (1 P3). Good to merge.
Verified: the T6/T6b split in tests/test_adr_rubric_frozen.py correctly branches on verdict, and the new 4-column Decision table in the ADR does not interfere with the weights-table regex (confirmed only the original 2-column Frozen-rubric table matches). Hand-checked the window-shortfall arithmetic (293h05m actual vs 324h04m declared = 30h59m shortfall = 90.4%) and the weighted score total (315/500) — both internally consistent. The 5 claimed mutants (uncited null, score 6, Decision without the verdict, Status Proposed, unknown verdict) each map to a real check in the updated test. One P3 nit posted inline (duplicated citation string, cosmetic only).
Records the #1280 soak verdict, ADOPT TEMPO / mctl-native only (315/500), in ADR 0001, the rubric and the runbook.
The window was shortened. The owner closed the soak on 2026-10-22 17:00Z for operator capacity, 30 h 59 min before the declared 2026-10-23 23:59Z (90.4% of the declared window). The new "Window amendment" section records this as a change to a declared parameter made after measurements had started, and the Decision opens with the same caveat.
Changes:
docs/adr/0001-agent-execution-trace-backend.md: Status is Accepted, plus the window amendment and the Decision (scores, decision-rule branch 3, the evidence). The Decision also keeps the findings that cut against the result: a bounded trace-by-id returned 5,092 of 6,930 in-window spans, which is unexplained (the ingest counters show no loss), and the verdict's own reads pushed the pod to 898 MiB after the cutoff.docs/adr/0001-rubric.yaml: the Tempo cells are scored, each with a citation. The other candidates are cited nulls ("not deployed in this round"). Adds the verdict and averdict_record.tests/test_adr_rubric_frozen.py: the freeze (T6) still holds while the verdict is null. Once a verdict is set, T6b applies instead: every cell has a citation, scores are integers 0–5, the ADR is Accepted with no UNFILLED marker, and the Decision names a permitted verdict. Mutation-checked with 5 mutants (uncited null, score 6, Decision without the verdict, Status Proposed, unknown verdict); each one fails the test.docs/runbooks/tracing-bake-off.md: what actually ran, the scored §8 contract and the read-path lessons.The measurements are in #1280 (comment).
Not in this PR: the teardown (runbook §9–10, due by 2026-10-24) and the adoption plan, which needs owner approval. Each is its own PR.
Refs #1280
🤖 Generated with Claude Code