fix: AI-DLC speaks in your project terms, not its own (#682) (2.5.57) - #707
Conversation
4e0a094 to
5ca15d0
Compare
leandrodamascena
left a comment
There was a problem hiding this comment.
Two issues need correction before merge:
-
[P1] Kiro IDE upgrade leaves duplicate audit hooks.
docs/guide/harnesses/kiro-ide.md:45-52sayscp -R dist/kiro-ide/.kiro/. your-project/.kiro/works for upgrades, but an overlay copy does not delete renamed files. An existingaidlc-audit-logger.jsonremains besideaidlc-write-audit-log.json; both register the samePostToolUseaudit-and-sensorsaction, so every artifact write is audited and sensor-checked twice. Please add an explicit migration/removal step for the old registration. -
[P2] The renamed intent command remains in the reference docs.
docs/reference/06-hooks-and-tools.md:224,418-421,529-530still documentsintent-birth, whiledocs/reference/17-skill-system.md:50,103referencesinit --scopeand the oldintent-birthmove. Please update these executable command references tointent-create.
Validation: bun run check, git diff --check origin/v2...HEAD, and 93 focused assertions passed.
|
Addressed the review findings in 184c669:
Validation: |
leandrodamascena
left a comment
There was a problem hiding this comment.
The latest commit resolves the previously blocking Kiro IDE upgrade issue and updates the retired intent-birth references. The cleanup is safe, idempotent, and correctly runs before the overlay copy. Focused tests, package parity, typecheck, and lint pass.
Approving with one non-blocking documentation follow-up: docs/guide/harnesses/kiro-ide.md still uses the retired names block, aidlc-mint, and aidlc-block at lines 118, 124, and 126. These should be updated to enforce-approval-gate, aidlc-record-human-turn, and aidlc-enforce-approval-gate.
The branch also needs a careful rebase onto current v2 2.5.55, preserving the recently merged reviewer-class and Unit-lifecycle behavior, followed by regenerated distributions and CI.
184c669 to
8554e37
Compare
fix: AI-DLC speaks in your project's terms, not its own (#682)
Closes #682.
Supersedes the three per-effort drafts: folder cleanup, hook renames, and the
voice work now ship as one PR from one branch.
The three complaints
Issue 682 reported three things. They turned out to share a cause: the framework
described itself to the user instead of describing their work.
Reverse Engineering (stage 2.1)."
the hook filename is the only label a waiting user sees, and
aidlc-sync-statuslineactively misled: it reconciles the Current Stage inaidlc-state.md, while the statusline is a separate hook that reads that file.bugfixrecordwas created with an empty
ideation/andoperation/beside the folders itwould really use, reading as planned-then-abandoned work.
What ships for each
Folders (complaint 3). Creating an intent now makes one artifact folder per
phase the scope actually runs, plus
verification/. Abugfixorrefactorrecord has no
ideation/oroperation/at all;featureandenterprisestill have all five. The folders created and the phases reported
PHASE_SKIPPEDnow derive from one place, so they always agree, and the dead
tools/data/scaffold/tree (26 empty directories no code read) is gone.Names (complaint 2). Seven hooks renamed to say what they do
(
aidlc-sync-statuslinetoaidlc-sync-workflow-state,aidlc-stoptoaidlc-continue-workflow, and five more), with the doctor heartbeats andhooks-health files following.
aidlc-utility intent-birthis nowintent-create, and the workspace formaidlc intent birthis nowaidlc intent create. Blocked-action messages carry an[aidlc]prefix so aline on stderr is attributable.
Voice at the asked surfaces (complaint 1, first pass). Scope questions,
approval gates, completion templates, the composer's plan proposal, error and
refusal strings, and the onboarding doc rewritten in plain developer terms, with
a "Talking to the user" voice contract in the stage protocol every harness loads.
The finding that changed this PR
Exploratory TUI testing of the merged branch found the first pass had fixed
every surface the framework asks on, and missed the one it speaks on. The
scope question, the approval gate, and the composer's plan lead all read plainly.
The free narration between tool calls did not:
That is issue 682's literal complaint, still present after a full voice pass.
Root cause: the model echoes the register of its own instructions. A single
voice contract cannot outweigh its container: one
SKILL.mdused "engine" 40times, "directive" 79, "dispatch" 29, "conductor" 12. The contract banned words
that the surrounding document headlined.
The first attempt at a fix was to flatten that vocabulary into plain language.
That was rejected and reverted (
d17599dc, reverted by81311e60): the internaloperating language is tested capital, and rewording it churns a large tested
surface to solve a presentation problem. The narration fix had to be additive
over unchanged internal language.
How it is fixed
Four candidate designs were built as spikes and measured by a shared live probe,
then composed:
narrationstring, authored where the facts already are. The engine knows deterministically
which stage this is, what scope resolved, and what comes next; the assistant
relays that sentence instead of inferring a description from the mechanism it
can see. Presentation only: it carries no routing meaning, any kind may omit
it, and dropping it changes nothing about what the framework does. This is the
same move the framework already made for rule delivery, where prose compliance
failed and deterministic injection replaced it.
No play-by-play, no naming the tool about to run, no reading a field back.
that no directive reaches (ensemble hand-offs, the review step), naming people
by trade rather than agent filename.
carrying internals: the rules-bundle header now says what the rules are rather
than which component resolved them, and the
§12a-style citations are gonefrom the sentences that leaked them.
opens the workflow description with "The orchestrator determines scope". It
reads "Scope is set from your command, or auto-detected from what you
describe."
Before and after, from the TUI ship gate:
The second finding: Construction
With the above in place, a full live end-to-end run (three hours, driven to
COMPLETEon anmvpintent) was the first evidence covering the wholelifecycle rather than its opening. It showed the contract holding through
Ideation and Inception at 6.9% of narration lines carrying a denied term, and
then regressing fourfold inside Construction, to 26.9%.
The cause was structural, not a wording miss. Construction repeats the same
stage once per unit of work, and the engine deleted the narration on every
directive carrying a unit. That was a deliberate choice: one line per unit
looked like noise, and the stage-closing directive was expected to still speak.
Neither held. The settle directive carries a unit too, so the whole stage ran
mute, and a mute moment is one the model fills from what is in front of it:
Two methodology nouns also reached the screen undefined. "Bolt" has a definition
in the delivery-planning stage file, but it sits in a block addressed to the
model, so it never rendered; and the walking-skeleton question asked whether to
"run the skeleton ceremony first", which is the framework's own phrasing for how
it classifies the answer, not a question a developer can answer.
This round closes both. Building passes now carry one authored line naming
the unit and the stage; the settle pass stays silent because the approval gate
owns that turn. The quiet rule gained an explicit Construction clause, in the
protocol and byte-identically in all five SKILLs, making the point that a plain
retelling of loop bookkeeping is not an improvement on it: the user has no
iteration and no gate boolean, so there is nothing in it to tell them. Both
nouns are glossed at first mention, in the sentence that uses them. And the
substitution the run showed six times at rule-loading boundaries ("Continuing to
the stage directive") is closed at its source: the instruction already forbade a
progress message, so it now also forbids putting a sentence of your own where
the progress message would have gone.
One asymmetry is deliberate and pinned as such: "skeleton ceremony" survives in
memory/org.md, because the stance classifier matches on that prose(
core/aidlc-common/conductor.md). What changed is the question built from it.Evidence
Live probe (assistant TEXT only, internal-vocabulary hits per 1000 words,
three prompts x three runs per arm):
Baseline 3.87 to a 0.5-1.0 band for the layered designs, a 4x to 7x reduction.
The seams (p1, p3) reach 0.00 in the composite: narration alone already
near-silenced them (0.34, 0.64) while p2 got worse (11.25), which is what
identified the leaks as living inside stages rather than between them and drove
the SAY-line layer. Best measured interior result is p2 0.98, from 10.66.
Honest caveat: at three runs per arm these are Poisson counts on small
samples. The finalist arms (voicepack2 0.54, composite2 0.58, composite3 0.72,
composite 0.96) are statistically indistinguishable from each other. The
baseline-versus-layered gap is the result that survives the sample size; the
ranking within the finalists does not, and the composite was chosen on design
grounds (every moment has a carrier) rather than on its margin.
TUI ship gate: sessions S2 and S3 rendered zero internal-vocabulary
leaks in what reached the screen.
Construction live drive (one SDK session seeded at the Construction
boundary with a two-unit Bolt DAG, project seeded outside the repo, driven
through the first unit's design stage to its confirmation gate). Every phrase
the three-hour run produced at these beats is now absent, and the authored line
arrives verbatim as the first thing said:
Counts over the captured assistant text:
gate false0,gate true0,build-artifacts pass0,per-unit iteration0,produces list0,Continuing to the stage directive0, plusdirective0,steering0,transport0,dispatch0,conductor0,harness0. The single "engine" hitis the user's own counting engine, not the framework's.
Two caveats kept deliberately visible. A first attempt at this drive hit an
unrelated fixture error (my seeded units artifact used a
units fence where the parser requiresyaml), and the error path the assistant then took spentseveral turns reading engine source, during which it did narrate framework
internals; that recovery path is outside what this round changed but is honest
evidence that a long repair detour still pulls the register back. On that same
degraded run the load-steering substitution appeared once rather than six times.
On the clean path it is zero.
Suite:
bun run typecheckclean,bun run checkclean over 552 files,bun scripts/package.ts --checkin sync across all five harness trees.smoke+unit and integration evidence in the run notes below.
Scope: measured on Claude, shipped to five
The probe and the TUI ship gate both ran against the Claude harness. The
other four (Kiro CLI, Kiro IDE, Codex, opencode) receive the identical
instruction prose and identical engine behaviour, but are not validated live in
this PR.
Worth stating plainly because it was nearly a real bug: the narration layer first
landed in the Claude
SKILL.mdonly, while the engine emitsnarrationfor everyharness. On the other four the field was emitted and silently dropped, leaving
them with exactly the behaviour this PR exists to fix, and nothing in the suite
went red. The rule is now ported to all five and pinned by
t181in two ways:the load-bearing tokens must be present in every harness SKILL, and the whole
rule must be byte-aligned across them so a per-harness reword is drift rather
than silent divergence. Both assertions were mutation-verified.
Recommended follow-up: a Kiro TUI spot-check to confirm the relay behaves
live on a second harness.
Residual risks
for minutes with no prose. The tool-call activity is visible in the UI, and
over-narration was the reported bug, so this is the intended trade. It wants
real-user feedback rather than another probe.
stage-protocol.md:769still reads "The orchestrator determinesappropriate depth based on scope selection." That is instruction prose, not
user-visible, and it stays as-is under the frozen-language decision. Flagged
because a future pass may want it.
The single-Bolt journey is validated end to end, and the per-unit line was
verified on the emitted directives for a two-unit DAG. What no live run has
crossed is the ladder prompt after Bolt 1 ships, the autonomous swarm fan-out,
and the per-unit worktrees. Those are exactly where the framework has the most
internal state to be tempted into narrating, and the quiet rule now covering
them is prose rather than an engine-authored line, so it carries the weaker of
the two guarantees this PR offers. A single mvp intent with three or more units
of work, driven through the ladder into an autonomous batch, is the missing
evidence.
Known exclusions (pre-existing, not from this branch)
t193-compose-report-journeyis filed as [Bug]: composer routes bug-shaped scan reports to a custom scope instead of matching stock bugfix (t193 red on clean v2) #704: the composersynthesizes a custom scope instead of matching stock
bugfix. Reproduced 4/4on a clean
v2base with the same signature, so it is not branch-attributable.Fix direction is a deterministic nearest-stock flip-diff.
t-journey-workspace.sdkis a flake: it fails and passes across runs, andits mechanism (a drifted cwd making the agent retry a completion report) is
unrelated to any line here.
t27standalone has one pre-existing red ("Awaiting your approval for [?]stage"), verified failing identically on the untouched base.
t197-compose-chat-inflight.sdkfailed on the integration tier and wasre-run alone on the untouched base at
4bbb5ddb, where it fails with the sameassertion. Pre-existing, not branch-attributable.
t163-reaper-steal-raceandt72-stage-reverse-engineeringboth failedin the parallel tier and both passed green-alone on re-run, which is the
flake signature (t163 is a concurrent-lock race; t72 is a live agent at 386s).
t100carries three pre-existing reds in itsadvance/approvememory_pathblock, verified failing identically on the untouched base beforethis round touched the file. The template-wording assertions this round moved
all pass.
Upgrade
Re-copy your
dist/<harness>/shell into the project. Mandatory, notoptional: the old hook files and their
settings.jsonentries reference theold names, so a stale copy keeps firing the old files alongside the new ones.
Re-copying also drops the dead scaffold directories and picks up the new prose.
Codex users additionally re-run the hook-trust pre-seed
(
bun scripts/package.ts codex trust --project <abs-dir>), because the trusthash covers the adapter command string and every renamed target changes it.
Existing intent records are untouched: nothing is ever deleted, so a record
already carrying all five phase folders keeps them.
Version 2.5.37, one consolidated CHANGELOG entry.