Repository navigation
record-and-heal: armed --save-script repair records the agent's read-only diagnostics into the healed script (can introduce new divergences) #1271
Description
Summary
During an armed --save-script record-and-heal repair segment, the agent's read-only diagnostic
commands (snapshot -i, get attrs, find, is) are recorded into the healed script alongside the
corrective actions. This pollutes the healed artifact, and in one measured case a recorded diagnostic
introduced a new, self-inflicted divergence on fresh replay.
Found in the wave-3 E3 repair-economics experiment (single-field-rename drift on iOS test-app).
Evidence
Two independent sonnet trials both used the documented --save-script/--from heal-by-doing flow
correctly, and both still required a manual .ad edit afterward solely to remove recorded diagnostics:
- sonnet-1: three extraneous
get attrs …steps (from triage inspection) were captured into the healed
script; harmless but not part of the flow — hand-removed. - sonnet-2: a
get attrs @e13run during triage was captured; on a fresh replay its recorded
ancestry did not hold and it caused a second divergence (identity-mismatch: recorded ancestry
scrollview/"Checkout form"vs observedother/"Legal name"). Hand-removed to finalize.
Net: across all 4 E3 trials, 0 produced a clean healed script hands-off — the flow foot-guns even the
agent that follows it. --no-record exists but neither agent discovered it for reads mid-repair.
Why it matters
The record-and-heal flow's value is a clean, replayable healed script produced without hand-editing. Today
it doesn't deliver that: an agent that inspects the screen to find the corrected target (the expected
behavior) pollutes its own output, and the pollution can be worse than cosmetic (a recorded read can carry
state-dependent identity that re-diverges).
Design options (maintainer call — not a silent patch)
- Cheap / hint-layer: the divergence
repairHinttext (andhelp workflow) should teach that
inspections during an armed repair must use--no-record. Zero behavior change; just stops the agent
foot-gunning itself. Aligns with "tune the hint, not the design." - Real / ADR-0012 decision-6 amendment: decide whether read-only commands (
snapshot/get/is/
find) recorded inside a repair segment should default out of the healed script (they never mutate
state, so they're arguably diagnostics by definition, not flow steps). This changes what the heal
transaction commits and is a semantic decision about the record-and-heal contract — hence an ADR
amendment, not a patch.
Option 2 is the more complete fix; option 1 is a safe interim. A related sub-finding worth its own look:
the sonnet-2 case is also a state-dependent-identity data point on iOS (a field's recorded ancestry
differed between mid-repair and fresh replay) — in mild tension with the E1 iOS FDR-0 result, whose flows
didn't contain a get-on-a-just-interacted-field step. Worth an autopsy (filled vs focused vs scroll state).
Activity
thymikee commentedon Jul 15, 2026
Maintainer triage: this is a record-and-heal contract decision, so
needs-triageis intentional and it should not be assigned as an AFK implementation yet.Recommended two-stage handling:
repairHintandhelp workflowto say that read-only diagnostics during an armed repair must use--no-record. Add a focused command-planning/SkillGym regression so agents discover this without hand-editing the healed script.The interim guidance is low risk and independently useful. Default exclusion is the more complete behavior, but it should not ship as an undocumented recording-contract change.
fix: warn agents that armed-repair diagnostic reads are recorded too (#…
thymikee commentedon Jul 16, 2026
Stage 1 shipped in #1287 (merged). This issue stays open to track stage 2.
What landed:
repairHintnow appends a diagnostics clause — "Read-only inspection while armed (snapshot -i, get attrs, find, is) is recorded too — use--no-recordon those commands or they will land in the healed script." — gated strictly onresume.repairSessionHeld === true, so it renders only on divergences from a repair-armed (--save-script) replay and is silent otherwise.help workflow's heal-by-doing section (distinct from the pre-existing--no-recordmentions, which are about fixing app state, not inspection reads).record-and-heal/state-repair/caution/manual) + a SkillGym case. The SkillGym case was live-validated: with a facts-only contract it reproduced the bug (haiku emitted unrecordedget attrsreads); with the clause present it passed 3/3 across haiku and codex-mini. So the guidance demonstrably changes agent behavior — which was the point of the interim stage.No recording behavior or defaults were changed.
Stage 2 — still open, still ADR-gated
Per the triage, the ADR-0012 amendment must define: (1) which read-only commands are excluded from a repair transaction, (2) whether explicit recording remains possible (and how it's requested), (3) how the rule projects across CLI / Node / MCP surfaces — with regression coverage proving diagnostic reads are omitted while corrective actions remain, and that the healed script replays cleanly.
One design wrinkle worth capturing before the amendment is drafted: "read-only" is not a clean discriminator on its own. A corrective action can itself be a read — in the wave-3 E3 drift the diverged step was a
get, so the corrective step is a recordedgetthat must land in the heal. A blanket "exclude reads from a repair segment" would silently drop it and produce a heal missing the very step it repaired. The amendment therefore needs to separate "the read that replaces the diverged step" from "reads used to locate the target" — e.g. by position relative to the resume, or an explicit opt-in — rather than excluding by command class alone.test(integration): cover --record with a provider-backed repair-segme…
fix(replay): default-exclude observation-only reads from repair heals…
test(integration): cover --record with a provider-backed repair-segme…
fix(replay): key the repair-segment exclusion on provenance, scope --…
test(daemon): pin that wire-supplied `internal` never reaches a daemo…
fix(replay): default-exclude observation-only reads from repair heals…
test(integration): cover --record with a provider-backed repair-segme…
fix(replay): key the repair-segment exclusion on provenance, scope --…
test(daemon): pin that wire-supplied `internal` never reaches a daemo…
fix(replay): default-exclude observation-only reads from repair heals…