Skip to content

v1.91.15.0 v1.91.15.0: Opus 5.5 prompt cleanup, behavior-level skill tests, transcript consent - #3017

Merged
garrytan merged 202 commits into
mainfrom
garrytan/opus-5-5-prompt-audit
Oct 3, 2026
Merged

garrytan merged 202 commits into
mainfrom
garrytan/opus-5-5-prompt-audit

Conversation

@garrytan

@garrytan garrytan commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Summary

This PR applies the findings of Anthropic's /claude-api prompt-audit (target claude-opus-5-5, Claude Code 2.1.287, 11 parallel slices) across gstack's skill templates, resolvers, model overlays, CLAUDE.md and the code that builds model requests. A follow-up /test-audit sweep, run as 14 slices, covered the tests that pin skill wording. /autoplan reviewed the plan with Claude and Codex (gpt-6-astra), with an Eng re-review for the test workstream.

gstack had almost none of the classic old-model scaffolding: no "think step by step", no scratchpads, and no API calls that disable thinking or force tool use. What it did have falls into three groups:

  • instructions that contradicted the code
  • CRITICAL/MUST stacks and model-computed point scores that current models over-apply
  • test files that pinned thousands of exact sentences, which kept that shouting in place

What changed

Stale facts and broken commands (W2). These are fixed in prompt text where the agent acts on it. Examples:

  • the router's empty telemetry session ID
  • design board flow and reload JSON
  • Pretext API signatures
  • a nonexistent make-pdf flag
  • the land-and-deploy readiness gate (bare bun test, legacy eval store)
  • stale /ship step numbers, hardcoded main, false "Mac only" claims
  • terminal-agent tab fields
  • spec pseudo-flags
  • autoplan skip headings
  • 23 literal ~/.gstack paths that bypassed GSTACK_STATE_ROOT

Normal-volume instructions (W3). Pressure stacks, thoroughness boosters, fixed progress cadences, point arithmetic and incident archaeology are rewritten as plain rules, keeping every constraint and its reason. Safety rules ("UC2 gates") changed wording only where an eval proves the behavior holds:

  • document-release's CHANGELOG-preservation and VERSION-ask rules were rewritten after the fixed document-release eval passed on both the old and the new text.
  • CODEX_BOUNDARY, the ship IRON LAW, design-review stop conditions, the pair-agent relay rule, codex consult-mode embedding and ios-qa DEMO MODE keep their wording. TODOS.md names the eval each one needs.

Model routing (W1). No new overlay: Claude hosts always render claude.md. resolveModel lowercases its input and strips [1m]-style markers. Opus 4.7, Opus 4.8 and Sonnet 5 keep their pinned profiles only for the bare, dated and -latest IDs. Point releases (claude-opus-5-5, claude-sonnet-5-5, …) fall to claude.

Transcript consent (W4). Transcripts are ingested only for transcript_ingest_mode recent or all. Absent, off, legacy and unknown values all skip transcripts.

  • Consent is read once per run.
  • A notice prints even under --quiet.
  • The stage record and checkpoint identity include the consent policy.
  • brain-sync holds staged transcript pages.
  • The config validator accepts only recent|all|off.
  • A setup-gbrain/memory.md#transcripts doc section is added.

Settled conflicts (W5).

  • PROACTIVE=false means no suggestions at all.
  • /sync-gbrain is the only writer of the CLAUDE.md search block.
  • /ship defers to the checklist's critical categories.
  • plan-design-review runs Step 0 before mockups.
  • DX triage keeps every pass and reports blockers only.
  • When DESIGN.md exists, it decides what design-shotgun varies.
  • ETHOS.md's claim about preamble injection is corrected, with Garry's approval.

Tests check behavior, not sentences (W7).

  • Harness freeze. Evals stop coaching the answer:
    • The document-release prompt no longer says "don't overwrite CHANGELOG".
    • The routing fixture is built from the routing text gstack ships.
    • Graders check outcomes, not exact phrases or emoji.
    • The dedicated-tools overlay contract v3 passes on correct output plus at least 20% fewer Bash calls.
    • The plan-artifact case stays inside its hermetic state root.
  • Prose-pin ratchets converted to structural checks. These kept order, routing tables, machine-read markers and case-insensitive safety lines. The largest:
    • ship-workflow-clarity: 764 → 286 expects
    • skill-ceo-section-ordering: 535 → 136 toContain
    • plan-review-cases: 592 → 366
  • Retirements. Six paid cases that only had a model summarize SKILL.md are retired, along with duplicate checks such as spec-template-sync and the 75 KB literal. Their registrations are cleaned up too.
  • New free tests.
    • test/instruction-facts.test.ts pins CLAUDE.md facts, /ship step references and state-root literals to the code.
    • A config-key reader tripwire covers config keys.
    • CLAUDE.md now defines when prompt bytes are a contract.

Validation

  • Free suite: bun run test:ubicloud (Bun 1.4.0, 17 shards) at ff245b11a passed all 24,058 tests.
  • Mutation passes: each converted test file got a scripted mutation pass, about 470 mutations across the F1 and F2 logs, with 0 unexpected outcomes.
    • Reordering a kept step, deleting a marker or deleting a safety line turns the test red.
    • Rewording a dropped sentence leaves it green.
    • F1 also found three mutations the old tests missed.
  • Paid evals, run locally:
Case Before (old templates, new harness) After (this branch)
document-release (UC2 gate) PASS PASS
plan-devex-review plan-mode (2 tests) PASS PASS
office-hours auto-mode PASS PASS
dedicated-tools overlay, Opus 4.7 PASS PASS
dedicated-tools overlay, Sonnet 4.6 PASS PASS
dedicated-tools overlay, Opus 5.5 (one-off, unregistered) PASS PASS
  • Typecheck: bun run typecheck reports 0 errors, and typecheck:test reports no new diagnostics.
  • Credential scan: the pre-push credential guard is enabled, and a per-file gstack-redact scan of the branch diff found only one hit, a git index hash matching a card pattern.

Not in this PR (see TODOS.md)

  • Judge-prompt cleanup (llm-judge.ts "ONLY valid JSON" and the inline judge schemas in skill-llm-eval.test.ts). The plan requires a calibration corpus with an old-vs-old flip rate first, and none exists yet.
  • Registering the Opus 5.5 dedicated-tools case as a periodic case. It moves pinned CI census walls and floors.
  • A fast-PR CEO probe. plan-ceo-review-plan-mode fails the one-owning-file planner rule.
  • Ratcheting the parity caps back down after merge.

Local-only test failures (pass in CI)

These failed on Capy machines and also fail the same way on the base f30b7b7. Their causes are the forced git identity, PTY behavior under Bun 1.3.14, and the build:cso prerequisite.

Open in Capy

resolveModel lowercases input and strips one trailing [...] context marker.
Opus 4.7, Opus 4.8 and Sonnet 5 keep their pinned overlay only for the bare
ID, the bare ID plus an 8-digit date, and the bare ID plus -latest; any other
suffix (point releases such as claude-sonnet-5-5, claude-opus-4-7-1) falls to
the generic claude overlay. claude-opus-5-5 routes to claude explicitly (no new
overlay, no new ALL_MODEL_NAMES entry). Fable keeps its prefix match while
CLAUDE_FRONTIER_EVAL_MODEL is claude-fable-5-1, guarded by a test.

Tests: one test/model-overlays.test.ts case per shape in the plan, plus the
lowercase / [1m] marker / family-name cases and the Fable guard.
…mpt-byte rule)

Adds the gstack-repo definition to the Testing section at normal volume:
prompt bytes are a contract only when software reads them or a recorded eval
shows the wording matters. Amends the test-value-bar line so 'prompt bytes'
refers to that definition. scripts/resolvers/test-value.ts and
docs/test-value-bar.md are unchanged.
$_SESSION_ID is never set in the router's bash block since
bin/gstack-skill-start took over session bookkeeping, so every /gstack route
event logged an empty session id. Use the literal SESSION_ID placeholder and
tell the model to substitute the echoed value, matching the completion-status
telemetry block. No W7a pins on the changed line.
H2: TREE_MUTATING is not empty; it still holds bootstrap-retention.
H3: catalog ceiling is CATALOG_BUDGET_TOKEN_EQUIVALENTS (1,194), not 1,171.
H4: detach timeouts point at package.json's --timeout values (gate is 49320s).
H5: Writing Style lives in preamble/generate-writing-style.ts and the jargon
list is Read at runtime, not baked at generation time.
No W7a pins on the changed lines.
The list has 77 terms, not 80+, and the model Reads the file anyway. No W7a
pins on the changed text.
…host may lack (s01 M1)

The nudge named Glob and Grep, which some Claude hosts do not expose. It now
names the host's dedicated file tools and its search tools when it has them,
keeping the prefer-dedicated-tools behavior the claude-dedicated-tools-vs-bash
overlay eval measures. Updates the prose comment at
test/gen-skill-docs.test.ts:643 (comment only, no assertion change). The
matching comment in test/fixtures/overlay-nudges.ts is left to its owner.
…nce (s01 M2)

Context Health told the model to periodically write a tagged [PROGRESS]
summary. No code reads that tag; the line now says when to tell the user
(finishing a phase or changing direction). Loop detection stays.

W7a pin: test/terse-build.test.ts:103 pinned 'PROGRESS'; it now checks the
default build emits a non-empty Context Health body (terse still emits '').
Drops the issue number, the 43-of-44 statistic and the ALWAYS caps from the
Operational Self-Improvement step; keeps the rule that the review runs every
time and that an empty result is stated explicitly. No W7a pins on the
removed text.
…01 M4, CEO W4)

The 'don't give up' section told agents to poll a plain background task with
sleep 180 + TaskOutput, which the next section forbids for evals. Its one
durable rule (keep checking until the run finishes, report progress at each
check) now lives in the detach section's poll step, pointed at the
gstack-detach EXIT sentinel, at normal volume. No tests pin the removed text.
…tion Format (s01 M5)

The two quoted pseudo-briefs pinned a superseded question shape (RECOMMENDATION:
Choose X, a completeness score on the recommendation, no D<N>/ELI10/Net). The
triage step now gives the facts, the options with their completeness values,
and the recommended option as data, and defers the shape to the Format section.
Option labels that Step T4 and test/skill-validation.test.ts:2004-2016 depend
on are verbatim. No W7a pins on the removed text.
… me to run /X?' (W5.1)

The router forbade suggesting skills when PROACTIVE is false while the preamble
told the model to ask 'want me to run it?'. Decision: false means never
auto-invoke and never suggest, including by asking. The preamble line in
generate-upgrade-check.ts now matches the router, and both quote the value
as `false`, the way gstack-skill-start prints it. Restore with
gstack-config set proactive true.

W7a pin: test/gen-skill-docs.test.ts:4488 used the old quoted sentence as a
preamble-order marker; it now finds the PROACTIVE-false rule with a
quote-tolerant, case-insensitive pattern.
…reamble (W5.7)

Nothing injects ETHOS.md. Preambles carry condensed principles (completeness,
voice, search before building) and the Search Before Building section links to
this file. Edit approved by Garry for this PR.
…_ID (s07 H1, H2, H3, M7, M10; s01 H1 office-hours note)

- Phase 4.5 points at Founder Resources instead of the nonexistent Beat 3.5 (H1).
- builder-journey.md is written to $GSTACK_STATE_ROOT, where it is opened (H2).
- builder-profile context query reads developer-profile.json, the file the
  migration leaves behind; drop meaningless tail (H3).
- Drop issue numbers from three headings (M7) and the MANDATORY/NOT optional
  markers on Phase 4 (M10).
- Handoff telemetry uses the echoed SESSION_ID placeholder; $_SESSION_ID is
  never set in a fresh shell.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…IDs (s07 H4, M1, M2, M3, M4, M5, M6)

- H4: Standard template heading no longer names nonexistent --bug/--feature/--refactor flags.
- M1: Phase 3 read-code-first gate at normal volume, constraint and reason kept.
- M2: drop the in-phase repeat of the read-first rule (Rules recap stays).
- M3: drop the duplicate audit-sink paragraph.
- M4: drop TTHW telemetry block and its never-set frontmatter keys.
- M5/M6: drop plan-review finding IDs from Phase 5 headings and the /ship handoff.

W7a pins: spec-template-invariants:314 (Mandatory/MUST) becomes a Phase 3
section check for the read-evidence-before-asking constraint; :203 matches
'raw spec must not be persisted' across line wraps so no duplicate paragraph
is needed to satisfy it.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…(s07 M9)

The self-regulation check keeps its cadence (every 5 fixes or after a revert),
its signals (reverts, unrelated files, sprawl, long runs, low-severity tail),
STOP-and-ask, and the 50-fix hard cap. The percentage table goes; the stop
threshold (two reverts, or one unrelated-file edit) stops at least as early.

W7a pins: qa-browser-preservation:220 drops 'WTF > 20%' and checks the
signals case-insensitively; skill-validation:1250 checks the test-only
exclusion rule instead of the heading name; carve-guards qa anchor
'WTF-LIKELIHOOD' becomes the '### 8f. Self-Regulation' heading.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…ck server state (s08 H1, H2, H3, H4, H7, M6)

- H1-H3: walkLineRanges, layoutNextLine and layoutWithLines signatures and
  return shapes now match design-html/vendor/pretext.js; patterns 2-4 rewritten.
- H4: inline the ES-module bundle in <script type="module"> with page code in
  the same module; no window.Pretext global exists.
- H7: remember the printed server URL and PID; later blocks restate them.
- M6: drop the 2-3 line cap on the refinement summary.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…e phrasing (s08 H5, M7, M9)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
… history (s08 H6, M1, M2, M3; W5.6)

- H6: the office-hours lookup block evals gstack-slug before using $SLUG.
- M1 + W5.6: with a DESIGN.md, variants keep its fonts and palette and vary
  layout and composition; without one they vary font, palette and layout.
  The swap-the-headline test and the 'Anti-convergence directive' heading
  (carve-guards anchor) stay.
- M2: drop the Claude Code version pin; run_in_background: false and its
  reason stay.
- M3: /tmp-then-cp rule and reason without the incident story.
G1 (replace executor subagents) deferred to TODOS.md per plan.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…audit (s08 M4, M10)

M5 (risk-percentage stop conditions) is a UC2 gate and keeps its wording.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…eason (s08 M8)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…09 H8, H9, H10, M10; W4 plan-tune)

W4: make the text true rather than change the hook. The consent copy and
scope now say never-ask questions are auto-decided with gstack's
recommendation by the question-preference hook (one-way doors still ask),
and the profile itself never changes defaults. H10 removes the nonexistent
--dismiss command. M10 strips cathedral/Layer/D-codes and keeps mechanisms.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…le (s09 H11, M1, M2, M3, M4, M5, M6)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…s09 H13, M7, M8, M9, M25)

H13 and M7 touch neighbouring lines and were merged by hand in that order.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…odes (s09 H12, M21, M22)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…e (s09 M20)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…s09 M23)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…ce (s09 M24)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
…s10 M1, M2)

The synthesis line keeps its Recommendation: <action> because <reason> shape.
M16 (consult-mode embed rule) is a UC2 gate and keeps its wording; the
outside-voice prompt bodies sent to Codex are unchanged.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.
bin/gstack-brain-context-load.ts resolves the {gstack_state_root}/ prefix
through gstack-paths (configuredStateGlob); the legacy ~ form only matched the
default root. The nine projects/analytics manifest globs in design-consultation,
design-shotgun, investigate, office-hours, plan-ceo-review and retro now use the
prefix, as does office-hours' builder-profile developer-profile.json glob
(plan-tune writes it under $GSTACK_STATE_ROOT).

Allowlist: drops the nine MANIFEST_GLOB rows and the now-unused MANIFEST_GLOB
and PENDING reasons; the header lists the remaining row kinds.
Tests: new brain-context-load case loads the shipped investigate and
office-hours queries against a configured root and proves a file only under
~/.gstack is not read.
@trunk-io

trunk-io Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

@github-actions github-actions Bot changed the title v1.91.15.0: Opus 5.5 prompt cleanup, behavior-level skill tests, transcript consent v1.91.15.0 v1.91.15.0: Opus 5.5 prompt cleanup, behavior-level skill tests, transcript consent Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

E2E Evals: ✅ PASS

[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 94/339 · behavior 0/28 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 274 · INFRA 0 · INCOMPLETE 0 · unattributed 22 · ACTION REQUIRED 0
  wall 10m50s · cost $84.12

102 executed, 16 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $84.12 rule/judge cost | reconcile exit: 0

Profile: pr / full-fallback; selected behaviors: 103, judges: 24. Deferred to scheduled/release coverage: 108 behaviors and 42 changed prompt files. Deferred checks did not run and receive no PR-pass credit.

Rule and judge shards
Shard Automated result Manual/unscored Executed Reused Status Cost
e2e-review-army/skill-e2e-review-army--review-army-perf-n-plus-one 1/1 0 1 0 ✅ $1
e2e/skill-e2e-shared-libs--shared-libs-review-revalidation 1/1 0 1 0 ✅ $4.42
e2e-session-intelligence/skill-e2e-session-intelligence 4/4 0 4 0 ✅ $0.82
e2e/skill-e2e-shared-libs-paths--shared-libs-review-index-flags 1/1 0 1 0 ✅ $2.33
e2e-triage/skill-e2e-triage 1/1 0 1 0 ✅ $0.35
e2e-review-army/skill-e2e-review-army--review-army-migration-safety 1/1 0 1 0 ✅ $0.96
e2e-retro/skill-e2e-retro 1/1 0 1 0 ✅ $1.03
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-timeout-unsettled 1/1 0 1 0 ✅ $0.54
e2e-skillify/skill-e2e-skillify 3/3 0 3 0 ✅ $2.4
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-launch-failure 1/1 0 1 0 ✅ $0.47
e2e-review-army/skill-e2e-review-army--review-army-quality-score 1/1 0 1 0 ✅ $0.4
e2e-qa-workflow/skill-e2e-qa-workflow 3/3 0 3 0 ✅ $3.52
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-current 1/1 0 1 0 ✅ $3.42
e2e-browse/skill-e2e-bws 6/6 0 6 0 ✅ $0.77
e2e/skill-e2e-shared-libs--shared-libs-review-lifecycle 1/1 0 1 0 ✅ $2.37
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-unavailable 1/1 0 1 0 ✅ $1.56
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-after 1/1 0 1 0 ✅ $0.89
e2e/skill-e2e-workflow 4/4 0 4 0 ✅ $2.5
e2e/skill-e2e-test-value 3/3 0 3 0 ✅ $2.29
llm-judge/skill-llm-eval 24/24 0 8 16 ✅ $0.48
e2e-qa-functional-fix/skill-e2e-qa-functional-fix 2/2 0 2 0 ✅ $2.41
e2e-qa-callers/skill-e2e-qa-callers--review-exploratory-small-cli 1/1 0 1 0 ✅ $1.77
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-late-input 1/1 0 1 0 ✅ $1.97
e2e-cso/skill-e2e-cso 1/1 0 1 0 ✅ $1.62
e2e/skill-e2e-shared-libs--shared-libs-unsupported-git 1/1 0 1 0 ✅ $0.58
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-failure 1/1 0 1 0 ✅ $1.88
e2e-qa-functional/skill-e2e-qa-functional 2/2 0 2 0 ✅ $2.34
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-completion 1/1 0 1 0 ✅ $3.87
e2e-design/skill-e2e-design--design-review-detector-shim 1/1 0 1 0 ✅ $0.35
e2e-review-army/skill-e2e-review-army--review-army-delivery-audit 1/1 0 1 0 ✅ $0.33
e2e/skill-e2e-coverage-audit 2/2 0 2 0 ✅ $0.83
e2e-review-army/skill-e2e-review-army--review-army-json-findings 1/1 0 1 0 ✅ $0.25
e2e/skill-e2e-investigate-owned-termination 2/2 0 2 0 ✅ $0.52
e2e-design/skill-e2e-design--plan-design-review-no-ui-scope 1/1 0 1 0 ✅ $0.33
e2e-third-party-actions/skill-e2e-third-party-actions 5/5 0 5 0 ✅ $1.2
e2e-learnings/skill-e2e-learnings 1/1 0 1 0 ✅ $0.27
e2e-plan-tune/skill-e2e-plan-tune 1/1 0 1 0 ✅ $0.37
e2e-design/skill-e2e-design--design-review-detector-shim-dom 1/1 0 1 0 ✅ $0.61
e2e-review-attribution/skill-e2e-review-attribution 3/3 0 3 0 ✅ $0.93
e2e/skill-e2e-plan--plan-review-report 1/1 0 1 0 ✅ $1.19
e2e/skill-e2e-investigate-owned-completion 1/1 0 1 0 ✅ $0.26
e2e/skill-e2e-ship-hook-consent 2/2 0 2 0 ✅ $0.6
e2e/skill-e2e-ship-hook-refresh 1/1 0 1 0 ✅ $0.35
e2e/skill-e2e-ship-skip 1/1 0 1 0 ✅ $0.54
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-marker 1/1 0 1 0 ✅ $0.48
e2e-docsync-spawned/skill-e2e-docsync-spawned 1/1 0 1 0 ✅ $1.22
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-stale-before 1/1 0 1 0 ✅ $0.91
e2e/skill-e2e-shared-libs--shared-libs-read-only 1/1 0 1 0 ✅ $0.94
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-late-result 1/1 0 1 0 ✅ $0.92
e2e-design/skill-e2e-design--design-review-plugin-handoff 1/1 0 1 0 ✅ $0.61
e2e-review/skill-e2e-review 2/2 0 2 0 ✅ $1.08
e2e/skill-e2e-shared-libs-paths--shared-libs-review-path-eligibility 1/1 0 1 0 ✅ $3.57
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-recovery 1/1 0 1 0 ✅ $0.84
e2e-hermetic/skill-e2e-hermetic-canary 2/2 0 2 0 ✅ $0.03
e2e/skill-e2e-shared-libs-paths--shared-libs-review-prior-coverage 1/1 0 1 0 ✅ $2.4
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-plan-checks 1/1 0 1 0 ✅ $1.96
e2e-qa-callers/skill-e2e-qa-callers--ship-exploratory-small-cli 1/1 0 1 0 ✅ $1.84
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-missing-asset 1/1 0 1 0 ✅ $1.17
e2e-ship-docsync/skill-e2e-ship-docsync--ship-docsync-store 1/1 0 1 0 ✅ $3.49
e2e-deploy/skill-e2e-deploy 6/6 0 6 0 ✅ $4.77
Fail-closed reconciliation
atory-unavailable
  slice 8  passed            129s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-stale-after
  slice 8  passed            146s  test/skill-e2e-shared-libs.test.ts#shared-libs-review-lifecycle
  slice 8  passed            261s  test/skill-e2e-plan-design-with-ui.test.ts
  slice 9  passed            143s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-late-result
  slice 9  passed            179s  test/skill-e2e-shared-libs-paths.test.ts#shared-libs-review-path-eligibility
  slice 9  passed            135s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-stale-before
  slice 9  passed            152s  test/skill-e2e-shared-libs.test.ts#shared-libs-read-only
  slice 9  passed            138s  test/skill-e2e-review.test.ts
  slice 9  passed             80s  test/skill-e2e-design.test.ts#design-review-plugin-handoff
  slice 10  passed            156s  test/skill-e2e-skillify.test.ts
  slice 10  passed            136s  test/skill-e2e-shared-libs-paths.test.ts#shared-libs-review-index-flags
  slice 10  passed             76s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-timeout-unsettled
  slice 10  passed            129s  test/skill-e2e-review-army.test.ts#review-army-migration-safety
  slice 10  passed            129s  test/skill-e2e-retro.test.ts
  slice 10  passed             59s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-launch-failure
  slice 10  passed             55s  test/skill-e2e-session-intelligence.test.ts
  slice 10  passed             34s  test/skill-e2e-triage.test.ts
  slice 11  passed            123s  test/skill-e2e-docsync-spawned.test.ts
  slice 11  passed             53s  test/skill-e2e-ship-docsync.test.ts#ship-docsync-missing-marker
  slice 11  passed             83s  test/skill-e2e-office-hours-auto-mode.test.ts
  slice 11  passed             76s  test/skill-e2e-third-party-actions.test.ts
  slice 11  passed             68s  test/skill-e2e-design.test.ts#design-review-detector-shim-dom
  slice 11  passed             74s  test/skill-e2e-ship-skip.test.ts
  slice 11  passed             79s  test/skill-e2e-review-attribution.test.ts
  slice 11  passed            226s  test/skill-e2e-plan.test.ts#plan-review-report
  slice 11  passed             45s  test/skill-e2e-ship-hook-refresh.test.ts
  slice 11  passed             50s  test/skill-e2e-ship-hook-consent.test.ts
  slice 11  passed             51s  test/skill-e2e-plan-tune.test.ts
  slice 11  passed             55s  test/skill-e2e-ask-user-question-format-compliance.test.ts
  slice 11  passed             28s  test/skill-e2e-investigate-owned-completion.test.ts
  slice 11  passed             33s  test/skill-e2e-learnings.test.ts
  slice 12  passed             67s  test/skill-e2e-coverage-audit.test.ts
  slice 12  passed             39s  test/skill-e2e-review-army.test.ts#review-army-delivery-audit
  slice 12  passed             47s  test/skill-e2e-investigate-owned-termination.test.ts
  slice 12  passed             45s  test/skill-e2e-design.test.ts#design-review-detector-shim
  slice 12  passed            108s  test/skill-e2e-plan-ceo-plan-mode.test.ts
  slice 12  passed             33s  test/skill-e2e-design.test.ts#plan-design-review-no-ui-scope
  slice 12  passed             35s  test/skill-e2e-review-army.test.ts#review-army-json-findings
[test:paid] collector final outcomes: 102 executed, 16 reused; 118 passed, 0 failed, 0 manual accepted (unscored; no score-cache credit) (118 attempt records from 61 collectors; every record counts)
[test:paid] report: ⚠ 1 shard(s) passed with EVERY test skipped — they verified nothing:
  ⚠ test/skill-routing-e2e.test.ts (11 skipped — external service missing or tier mismatch)
[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 94/339 · behavior 0/28 · judge 24/24 · quarantined 0 (0 failing the lane)
  SKIPPED 274 · INFRA 0 · INCOMPLETE 0 · unattributed 22 · ACTION REQUIRED 0
  wall 10m50s · cost $84.12
[test:paid] report: every planned shard accounted and passed

Sliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry.

@garrytan
garrytan merged commit 74512c2 into main Oct 3, 2026
55 checks passed
garrytan added a commit that referenced this pull request Oct 3, 2026
…elease metadata v1.91.16.0

Conflicts resolved by hand in templates, tests, CLAUDE.md, TODOS.md,
CHANGELOG.md and bin/gstack-config; generated SKILL.md/sections, the agents
digest and test/fixtures/context-budget.json regenerated. The eval codemod
re-applied to main's new gstack-paths/gstack-slug evals. Coverage-read rule:
bare echo separators after unknown output extend the unknown span (main's
pathspec CI shape). CLAUDE.md keeps the TREE_MUTATING, catalog-ceiling and
detach-timeout facts main's instruction-facts test pins.
levi85 pushed a commit to levi85/gstack that referenced this pull request Oct 14, 2026
…tests, transcript consent (garrytan#3017)

* fix(models): allowlist routing for model-pinned Claude families (W1)

resolveModel lowercases input and strips one trailing [...] context marker.
Opus 4.7, Opus 4.8 and Sonnet 5 keep their pinned overlay only for the bare
ID, the bare ID plus an 8-digit date, and the bare ID plus -latest; any other
suffix (point releases such as claude-sonnet-5-5, claude-opus-4-7-1) falls to
the generic claude overlay. claude-opus-5-5 routes to claude explicitly (no new
overlay, no new ALL_MODEL_NAMES entry). Fable keeps its prefix match while
CLAUDE_FRONTIER_EVAL_MODEL is claude-fable-5-1, guarded by a test.

Tests: one test/model-overlays.test.ts case per shape in the plan, plus the
lowercase / [1m] marker / family-name cases and the Fable guard.

* docs(CLAUDE.md): define when prompt bytes are a test contract (W7 prompt-byte rule)

Adds the gstack-repo definition to the Testing section at normal volume:
prompt bytes are a contract only when software reads them or a recorded eval
shows the wording matters. Amends the test-value-bar line so 'prompt bytes'
refers to that definition. scripts/resolvers/test-value.ts and
docs/test-value-bar.md are unchanged.

* fix(browse): sidebar tab hint names real tabs.json fields and how $B resolves (s11 H1, s11 M8, s11 M9)

* fix(router): route telemetry uses the echoed SESSION_ID (s01 H1)

$_SESSION_ID is never set in the router's bash block since
bin/gstack-skill-start took over session bookkeeping, so every /gstack route
event logged an empty session id. Use the literal SESSION_ID placeholder and
tell the model to substitute the echoed value, matching the completion-status
telemetry block. No W7a pins on the changed line.

* docs(CLAUDE.md): correct stale test-infra facts (s01 H2, H3, H4, H5)

H2: TREE_MUTATING is not empty; it still holds bootstrap-retention.
H3: catalog ceiling is CATALOG_BUDGET_TOKEN_EQUIVALENTS (1,194), not 1,171.
H4: detach timeouts point at package.json's --timeout values (gate is 49320s).
H5: Writing Style lives in preamble/generate-writing-style.ts and the jargon
list is Read at runtime, not baked at generation time.
No W7a pins on the changed lines.

* fix(preamble): drop the stale jargon-list count (s01 H6)

The list has 77 terms, not 80+, and the model Reads the file anyway. No W7a
pins on the changed text.

* fix(overlay): claude.md dedicated-tools nudge stops naming tools the host may lack (s01 M1)

The nudge named Glob and Grep, which some Claude hosts do not expose. It now
names the host's dedicated file tools and its search tools when it has them,
keeping the prefer-dedicated-tools behavior the claude-dedicated-tools-vs-bash
overlay eval measures. Updates the prose comment at
test/gen-skill-docs.test.ts:643 (comment only, no assertion change). The
matching comment in test/fixtures/overlay-nudges.ts is left to its owner.

* fix(preamble): progress notes on phase changes, not a [PROGRESS] cadence (s01 M2)

Context Health told the model to periodically write a tagged [PROGRESS]
summary. No code reads that tag; the line now says when to tell the user
(finishing a phase or changing direction). Loop detection stays.

W7a pin: test/terse-build.test.ts:103 pinned 'PROGRESS'; it now checks the
default build emits a non-empty Context Health body (terse still emits '').

* fix(preamble): learnings review without incident archaeology (s01 M3)

Drops the issue number, the 43-of-44 statistic and the ALWAYS caps from the
Operational Self-Improvement step; keeps the rule that the review runs every
time and that an empty result is stated explicitly. No W7a pins on the
removed text.

* docs(CLAUDE.md): fold 'Long-running tasks' into the detach section (s01 M4, CEO W4)

The 'don't give up' section told agents to poll a plain background task with
sleep 180 + TaskOutput, which the next section forbids for evals. Its one
durable rule (keep checking until the run finishes, report progress at each
check) now lives in the detach section's poll step, pointed at the
gstack-detach EXIT sentinel, at normal volume. No tests pin the removed text.

* fix(preamble): test-failure triage questions defer to the AskUserQuestion Format (s01 M5)

The two quoted pseudo-briefs pinned a superseded question shape (RECOMMENDATION:
Choose X, a completeness score on the recommendation, no D<N>/ELI10/Net). The
triage step now gives the facts, the options with their completeness values,
and the recommended option as data, and defers the shape to the Format section.
Option labels that Step T4 and test/skill-validation.test.ts:2004-2016 depend
on are verbatim. No W7a pins on the removed text.

* fix(proactive): PROACTIVE=false means no suggestions, including 'want me to run /X?' (W5.1)

The router forbade suggesting skills when PROACTIVE is false while the preamble
told the model to ask 'want me to run it?'. Decision: false means never
auto-invoke and never suggest, including by asking. The preamble line in
generate-upgrade-check.ts now matches the router, and both quote the value
as `false`, the way gstack-skill-start prints it. Restore with
gstack-config set proactive true.

W7a pin: test/gen-skill-docs.test.ts:4488 used the old quoted sentence as a
preamble-order marker; it now finds the PROACTIVE-false rule with a
quote-tolerant, case-insensitive pattern.

* docs(ETHOS.md): correct the claim that ETHOS is injected into every preamble (W5.7)

Nothing injects ETHOS.md. Preambles carry condensed principles (completeness,
voice, search before building) and the Search Before Building section links to
this file. Edit approved by Garry for this PR.

* office-hours: fix stale refs, state-root path, profile query, SESSION_ID (s07 H1, H2, H3, M7, M10; s01 H1 office-hours note)

- Phase 4.5 points at Founder Resources instead of the nonexistent Beat 3.5 (H1).
- builder-journey.md is written to $GSTACK_STATE_ROOT, where it is opened (H2).
- builder-profile context query reads developer-profile.json, the file the
  migration leaves behind; drop meaningless tail (H3).
- Drop issue numbers from three headings (M7) and the MANDATORY/NOT optional
  markers on Phase 4 (M10).
- Handoff telemetry uses the echoed SESSION_ID placeholder; $_SESSION_ID is
  never set in a fresh shell.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* spec: plain Phase 3 gate, drop duplicates, dead telemetry and review IDs (s07 H4, M1, M2, M3, M4, M5, M6)

- H4: Standard template heading no longer names nonexistent --bug/--feature/--refactor flags.
- M1: Phase 3 read-code-first gate at normal volume, constraint and reason kept.
- M2: drop the in-phase repeat of the read-first rule (Rules recap stays).
- M3: drop the duplicate audit-sink paragraph.
- M4: drop TTHW telemetry block and its never-set frontmatter keys.
- M5/M6: drop plan-review finding IDs from Phase 5 headings and the /ship handoff.

W7a pins: spec-template-invariants:314 (Mandatory/MUST) becomes a Phase 3
section check for the read-evidence-before-asking constraint; :203 matches
'raw spec must not be persisted' across line wraps so no duplicate paragraph
is needed to satisfy it.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* investigate: drop issue numbers from freeze-script comment (s07 M8)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* qa: replace WTF point table with named signals and a plain stop rule (s07 M9)

The self-regulation check keeps its cadence (every 5 fixes or after a revert),
its signals (reverts, unrelated files, sprawl, long runs, low-severity tail),
STOP-and-ask, and the 50-fix hard cap. The percentage table goes; the stop
threshold (two reverts, or one unrelated-file edit) stops at least as early.

W7a pins: qa-browser-preservation:220 drops 'WTF > 20%' and checks the
signals case-insensitively; skill-validation:1250 checks the test-only
exclusion rule instead of the heading name; carve-guards qa anchor
'WTF-LIKELIHOOD' becomes the '### 8f. Self-Regulation' heading.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* design-html: match Pretext docs to the vendored bundle; fix cross-block server state (s08 H1, H2, H3, H4, H7, M6)

- H1-H3: walkLineRanges, layoutNextLine and layoutWithLines signatures and
  return shapes now match design-html/vendor/pretext.js; patterns 2-4 rewritten.
- H4: inline the ES-module bundle in <script type="module"> with page code in
  the same module; no window.Pretext global exists.
- H7: remember the printed server URL and PID; later blocks restate them.
- M6: drop the 2-3 line cap on the refinement summary.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* make-pdf: drop nonexistent --no-syntax flag, trigger list and relative phrasing (s08 H5, M7, M9)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* design-shotgun: DESIGN.md decides what variants vary; fix $SLUG; drop history (s08 H6, M1, M2, M3; W5.6)

- H6: the office-hours lookup block evals gstack-slug before using $SLUG.
- M1 + W5.6: with a DESIGN.md, variants keep its fonts and palette and vary
  layout and composition; without one they vary font, palette and layout.
  The swap-the-headline test and the 'Anti-convergence directive' heading
  (carve-guards anchor) stay.
- M2: drop the Claude Code version pin; run_in_background: false and its
  reason stay.
- M3: /tmp-then-cp rule and reason without the incident story.
G1 (replace executor subagents) deferred to TODOS.md per plan.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* design-review: plain identity line; scope 'never read source' to the audit (s08 M4, M10)

M5 (risk-percentage stop conditions) is a UC2 gate and keeps its wording.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* design-consultation: plain CLAUDE.md design-system snippet with its reason (s08 M8)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* plan-tune: describe the never-ask hook truthfully; drop plan codes (s09 H8, H9, H10, M10; W4 plan-tune)

W4: make the text true rather than change the hook. The consent copy and
scope now say never-ask questions are auto-decided with gstack's
recommendation by the question-preference hook (one-way doors still ask),
and the profile itself never changes defaults. H10 removes the nonexistent
--dismiss command. M10 strips cathedral/Layer/D-codes and keeps mechanisms.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* context-save/restore: drop rename history; fix command-list branch rule (s09 H11, M1, M2, M3, M4, M5, M6)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* retro: correct what the skill writes; drop word cap and archaeology (s09 H13, M7, M8, M9, M25)

H13 and M7 touch neighbouring lines and were merged by hand in that order.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* health: compute the composite in code; fix example weights; drop D6 codes (s09 H12, M21, M22)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* gstack-upgrade: drop issue numbers and version history, keep rationale (s09 M20)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* canary: replace the 30-second/don't-over-analyze rule with the goal (s09 M23)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* benchmark: self-contained page discovery instead of a /canary reference (s09 M24)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* codex: drop judge vocabulary and the maintainer known-bad-list note (s10 M1, M2)

The synthesis line keeps its Recommendation: <action> because <reason> shape.
M16 (consult-mode embed rule) is a UC2 gate and keeps its wording; the
outside-voice prompt bodies sent to Codex are unchanged.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* document-generate: true redaction-fence behavior; drop thoroughness booster (s10 H8, M3)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* document-release: changelog sell test as three questions, not a 0-3 score (s10 M6)

The threshold stays (fewer than two answered = needs attention, not
replacement). M4/M5 (NEVER CLOBBER / NEVER BUMP) keep their wording; the
integrator owns the eval-gated rewrite.

W7a pin: workflow-excerpt:170 'Entries scoring <2 need attention, not
replacement' becomes a case-insensitive check that weak entries get
attention, not replacement.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* ios-clean/fix/sync: match the generator's real header, package shape and regen command (s10 H2, H3, H4, H5, M9, M12)

- H2: inventory greps the header the generator actually writes.
- H4/H5: remove the local .package(path:) entry and DebugBridge* product
  deps; delete the generated DebugBridge/ and DebugBridgeGenerated/ dirs so
  the Phase 3 check can pass.
- H3: regenerate accessors with /ios-sync (gstack-ios-qa-regen).
- M12: ios-fix writes the regression with the project's own test runner.
- M9: drop migration-relative sentence.
ios-qa DEMO MODE (M8) is a UC2 gate and keeps its wording.

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* setup-deploy: drop User-invocable boilerplate; say default branch, not main (s10 M10, M11)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* skillify: test staged skills directly; stop after Aside scrapes; drop roadmap notes (s10 H1, H6, H7, M14, M15)

Template-only commit: generated SKILL.md is expected red until the regeneration commit.

* W4: transcript ingest needs affirmative consent (recent|all)

normalizeTranscriptConsent() in gstack-memory-ingest reads
transcript_ingest_mode once per process. Only recent (90 days) and all
(all history) are consent; an absent key, off, legacy gate letters,
incremental and unknown values skip transcripts while other memory still
syncs. The ingest's default source set drops transcript without consent;
an explicit --sources list naming transcript overrides.

gstack-gbrain-sync reads consent once at sync start and pins a non-consented
run to an explicit source list, so a mid-run config change cannot add
transcripts. It prints the pinned skip notice on every such run (even
--quiet), records the normalized mode and skip reason in the memory stage,
restages an interrupted import when the consent policy changed, passes
--all-history on --full for mode all, and documents the default in --help.

Tests: new gstack-gbrain-sync-transcript-mode.test.ts; memory-ingest
fixtures seed consent; regression-2922's every-type case seeds consent.
Findings: s09 H16 (enforce the consent choice in code).

* W4: gstack-brain-sync holds staged transcript pages without consent

Queued transcripts/ pages stay queued and on disk unless
transcript_ingest_mode is recent or all; other classes still push.
Test: brain-sync.test.ts push-filter case (absent, off, legacy A, then recent).

* W4: gstack-config validates transcript_ingest_mode; set rejects empty and multi-line values

transcript_ingest_mode accepts only recent|all|off and leaves the stored
value unchanged otherwise. set rejects an empty value or one containing a
newline for every key instead of truncating it. The config header documents
the key; lookup_default keeps off.

* setup-gbrain, sync-gbrain: fix stale facts and strip plan archaeology

Applies s09 H1a, H1c (Mac-only wording; the installer supports Linux and
Windows), H2 (smoke test has no delete step), H3 (five invocation modes),
H4 (no CI grep test backs the telemetry claim), H7 (Step 4.5 does not exist;
the prompt is Path 4 step 4d), M11, M12, M13, M15, M18, M19 (plan codes,
issue numbers, version pins, rename story), M14 (restore-from-sync never
shipped), M16 (draft self-dialogue in the Supabase steps), M17 (passed
sunset date, manual merge into both sections).

W7a: retires the setup-gbrain-bin-invocation-paths 'now persists staged
transcripts' change-detector (t06 #17); the no-bare-invocation regex tests
keep the contract.

* W4: transcript consent prose in /setup-gbrain, /sync-gbrain and memory.md

The Step 7.5 gate is rewritten as a consent question (manual merge of s09 H5,
with H1b and H6): it checks gstack-config has, asks once with A) recent,
B) all history, E) never, names Claude Code and Codex sessions from every
repo-policy-allowed project and the destination brain, stores the mode value
(never the letter), asks yes/no when no transcripts exist, never asks
spawned or headless sessions, and drops the silent no-question bulk paths
and the false skill-start-hook claim (s09 H5c). Path 4 no longer stores off
on the user's behalf. /sync-gbrain gains Step 1.6 (ask when the key is absent
or holds a legacy value). Step 10 and the sync verdict show the mode.
setup-gbrain/memory.md gains #transcripts (values, sources, destination,
override precedence, one-time restage, held staged pages, already-ingested
pages stay).

W7a: setup-gbrain-bin-invocation-paths drops the silent-bulk and
preamble-hook pins (that prose is gone) for negative checks (no pre-answer
--bulk, no skill-start claim), loosens the gstack-config pin (t06 #18) to
the command without <choice>, and adds the DX 'gated on has' order check for
both skills.

* W5.2: /sync-gbrain is the only writer of the CLAUDE.md search-guidance block

setup-gbrain's claude-md-persist section stops writing its own
'## GBrain Search Guidance' block, which also removes the false
'auto-syncs on every gstack skill start' text (s09 H15; s09 H14 decided).
Step 10 offers to run /sync-gbrain now and names it otherwise. Test sweep:
only skill-e2e-sync-gbrain-readiness references the heading, and it checks
the block /sync-gbrain writes, so no test changes.

* E4: config-key reader tripwire in gstack-config-defaults.test.ts

Every key in bin/gstack-config's DEFAULTS table must be read by code
(gstack-config get/has <key>, a config-binary variable, a spawn argv, a
config-reader call, or an anchored ^key: pattern in bin/, lib/, scripts/,
setup or any .tmpl; wildcard keys by prefix) or appear in PROSE_ONLY_KEYS
with a reason. Run on the current tree first: every key has a reader, so
the seeded prose-only list is empty. Failure messages name the two fixes.
A synthetic-tree case checks each reader form and that docs prose does not
count. setup is scanned because timeline_stop_hook is read only there.

* W4: /open-gstack-browser asks before replacing a live browse daemon

Step 0 probes with BROWSE_NO_AUTOSTART=1 $B status (no daemon is booted by
the probe). No live daemon: clear Chromium profile locks and run plain
$B connect, which reaps orphans and refuses to replace a busy daemon. Already
headed: connect reports it. Live headless daemon: AskUserQuestion, and only
an explicit yes runs $B connect --force-restart; spawned or headless sessions
leave it running and print the stop line with the resolved browse binary.
The hand-rolled kill / kill -9 of the recorded PID and the browse.json
deletion are gone. Supersedes s10 M13.

Tests: new open-gstack-browser-liveness.test.ts (no kill of the recorded PID,
status probe before connect, --force-restart only after the yes branch,
profile-lock cleanup kept, spawned/headless stop line). The state-extraction
tripwire now covers only the port snippet, since the pid snippet is gone.

* fix(browse): untrusted-content warnings name both marker formats (s02 H5, s02 L2 / W7f)

The /browse warning named only the --- UNTRUSTED EXTERNAL CONTENT --- envelope
and the browser fallback only the ═══ UNTRUSTED WEB CONTENT ═══ one; $B prints
both (commands.ts wrapUntrustedContent, content-security.ts ENVELOPE_BEGIN).
Both surfaces now name both formats.

W7f pin: test/aside-driver.test.ts:347-350 no longer bans the EXTERNAL marker;
it reads both marker labels from browse/src/commands.ts and
browse/src/content-security.ts and requires both in the fallback and warning.
Negative control: reverting browse.ts turns the test red.

Parity cap (test/helpers/carve-guards.ts): qa maxSizeRatio 1.103 -> 1.104
(measured 1.1035); the second marker adds bytes to every skill that renders
the browser fallback.

expected red until the regeneration commit (generated SKILL.md not committed)

* fix(design): mockup board loop matches the daemon CLI (s02 H1, s02 H2, s02 H3)

- H1: $D compare --serve publishes to the design daemon and exits; read
  BOARD_URL from stderr, wait with AskUserQuestion, then read feedback.json /
  feedback-pending.json. The fallback no longer names a $D serve command the
  step never runs.
- H2: the reload POST uses the jq --arg form, so $_DESIGN_DIR expands (the
  single-quoted JSON sent the literal variable name).
- H3: Content & Microcopy lists 11 items, not 8.

No test pins the removed text. expected red until the regeneration commit

* design: name the defaults to avoid in native design-direction prompts; plain path rule (s02 H4a, s02 H4b, s02 M9)

- H4a/H4b: the design-consultation and design-sketch native subagent prompts
  now name the default looks to avoid (the three looks from the Hard Rules
  calibration plus the catalog entries the sibling outside-voice prompt
  already lists) and say to pick again if the first idea is one of them.
- M9: 'CRITICAL PATH RULE' -> 'Path rule'; the rule and its reason stay.

No test pins the changed text. Parity cap (test/helpers/carve-guards.ts):
design-consultation maxSizeRatio 1.085 -> 1.092 (measured 1.0911). expected red until the regeneration commit

* fix(outside-voice): preflight mode list no longer looks exhaustive (s02 M10)

The preflight also prints under_codex and under_current_harness; not_authed
exists only on the Codex path. 'every other mode skips them' covers all
non-ready modes. No test pins the old line. expected red until the regeneration commit

* outside-voice: drop booster sentences and plan-review decision IDs (s02 M1, s02 M7)

- M1: native adversarial subagent prompt drops 'Be adversarial. Be thorough.';
  the attacker framing, defect list and 'No compliments' output rule stay.
- M7: doc-review headings drop the D3 / T3B plan-review decision IDs (D<N> also
  collides with AskUserQuestion brief numbering).

Not changed: s02 M2 (outside adversarial prompt body sent to Codex/GPT: out of
scope per plan) and s02 M3 (CODEX_BOUNDARY: UC2 gate keeps its wording).
No test pins the removed text. expected red until the regeneration commit

* confidence: drop issue IDs and the contributor-only design-doc pointer (s02 M4a, s02 M4b)

- M4a: gate heading and FP-class intro lose '#1539' and the Django sprint
  measurement note; the gate rule and its four FP classes stay.
- M4b: remove the pointer to ~/.gstack-dev/plans/1539-framework-aware-review.md,
  a contributor-local file that never exists where /plan-eng-review and /ship run.

W7a pins (test/regression-1539-review-self-verify.test.ts):
- :34 toMatch(/#1539/) removed (cosmetic issue-number pin; the gate-header
  check stays).
- :64-67 'references the deferred design doc' replaced by a structural check
  that no .gstack-dev/plans/ path reaches the emitted prose.
expected red until the regeneration commit

* question-tuning: state why the summary is piped instead of citing #2024 (s02 M6)

No test pins the resolver text (#2024 appears only in test names/comments).
expected red until the regeneration commit

* fix(testing): shared TEST_BOOTSTRAP undoes only owned changes (s03 H1, s03 H2, s03 H3, s03 M1)

The shared {{TEST_BOOTSTRAP}} (/ship, /design-review) still told the agent to
'git checkout -- package.json package-lock.json', to 'revert all bootstrap
changes', to 'delete silently' a failing first test, and to stage 'all
bootstrap files'. The QA copy (qa/sections/test-bootstrap.md.tmpl) already
says undo only owned changes, never blanket-checkout, never silently delete a
valid red regression, and stop on unrelated staged edits. The shared copy now
matches it. This removes destructive behavior (UC2: lands regardless of eval).

W7g: test/qa-browser-preservation.test.ts gains a check that the shared copy
(ship, design-review) has no 'git checkout --', no silent deletion, no
blanket revert, and keeps the owned-change and staged-edits rules
(case-insensitive). Negative control: the old testing.ts turns it red.
Parity cap (test/helpers/carve-guards.ts): ship maxSizeRatio 1.404 -> 1.405
(measured 1.4044 at the branch tip, with the s03 M2/M3 wording).
expected red until the regeneration commit

* testing: coverage trace and test generation follow the value bar (s03 M2, s03 M3)

- M2: 'Every branch in this diagram needs a test' / 'Each step in the journey
  needs a test' contradicted COVERAGE_GOAL ('Test count is not a goal') in the
  same rendered audit. Branches and journey steps now need coverage that would
  catch a regression; the value bar decides new test vs extension vs covered.
- M3: 'Mock all external dependencies' -> the smallest native test, mocking
  only services unrelated to the behavior (matches the QA handoff in qa.ts and
  the audit's own E2E routing).

No test pins the removed text. expected red until the regeneration commit

* plan-gates: per-item confirmation states its reason, not an incident ID (s03 M4)

'VAS-449' meant nothing to the model and reached the user in the >5-items
warning. The rule now gives the reason (a blanket confirmation lets the user
click through without opening any file / lets unchecked items pass as
verified). The per-item confirmation rule is unchanged.

W7a pin: test/ship-plan-completion-invariants.test.ts:122 toMatch(/VAS-449/)
removed (cosmetic incident-ID pin); the per-item rule and blanket-confirm ban
checks on the same test stay. expected red until the regeneration commit

* plan-gates, review-dashboard: terminal-report rules without trait claims or 'prior versions' narrative (s03 M5, s03 M7)

- M5: exit gate drops 'will (correctly) reject it' and the 'Self-deception
  failure mode' aside; keeps the contract-violation sentence (pinned by
  test/gen-skill-docs.test.ts) and the body-prose-is-not-the-report rule.
- M7: report-write step states the rule and its reason (delete and append at
  EOF so the report stays last) instead of the history of how it broke.
No test pins the removed text. expected red until the regeneration commit

* spec-review: default anti-shortcut clause without incident narrative (s03 M6)

Drops 'the May 2026 transcript bug ... that's the bug, recognize it' and the
caps; keeps the rule: each non-trivial finding goes through AskUserQuestion
before it enters the plan, and only a zero-finding review skips asking. The
four skill-specific branches are unchanged.

W7a pin: test/skill-ceo-section-ordering.test.ts:868-873 pinned the SHA-256
of the default clause. It now checks, on every host, that the clause still
says the plan cannot replace the interactive review, routes findings through
AskUserQuestion, and lets only zero findings reach ExitPlanMode without
asking (case-insensitive), and that the clause is host-independent. The old
clause also passes; deleting the zero-findings rule turns it red. The DX
hash pin is unchanged.
expected red until the regeneration commit

* constants: foreground-dispatch note in current-state terms at normal volume (s03 M8)

Keeps every constraint (explicit run_in_background: false, Agent tool only,
never inline or as a Skill, fallback only after a dispatched subagent fails)
and the reason (fresh-context isolation); drops 'no longer', BACKGROUND/ONLY/
WRONG/FALLBACK caps. test/ship-control-flow.test.ts and
run-in-background-guidance.test.ts check the placeholder and the explicit
flag, not this prose. expected red until the regeneration commit

* tasks-section: inline the effort ratios for every phase (s03 M9)

The non-CEO/Eng branch said 'Effort uses the AI-compression table from
CLAUDE.md', but at runtime CLAUDE.md is the user's project file; the table
exists only in gstack's own repo. Every phase now renders the inline ratios
the CEO/Eng branch already used. No test pins the removed text.
expected red until the regeneration commit

* hosts, utility: model-neutral Claude co-author trailer (s03 M10, s11 M10)

The Claude host trailer and the resolver fallback (cursor, kiro, opencode,
slate) signed commits as 'Claude Opus 4.7' whatever model ran. Both now read
'Co-Authored-By: Claude <noreply@anthropic.com>' so the trailer cannot go
stale at the next model release (the s11 report offered this neutral form;
s03 M10's hunk uses it). document-release's release body renders the same
{{CO_AUTHOR_TRAILER}} and follows on regeneration.
test/host-config.test.ts:589 only requires 'Claude' and stays green. Goldens
move on regeneration (not committed). expected red until the regeneration commit

* changelog workflow: cross-check covers user-facing changes, not every commit (s03 M11)

'Every commit must map to at least one bullet' contradicted the same block's
voice rule (never mention contributor-facing details) and CLAUDE.md's entry
rule (the diff vs main, not how the branch got there). Every user-facing
change in the commit list must be represented; merges, version bumps and
mid-branch fixes need no bullet.

W7a pin: test/gen-skill-docs.test.ts:2038 no longer pins the old sentence; it
checks the Cross-check step still compares against the step-2 commit list
and scopes to user-facing changes (case-insensitive), without re-pinning the
new sentence. expected red until the regeneration commit

* plan-ceo-review, plan-eng-review: storage paths follow the state root (s04 F2, s04 F3)

The CEO storage policy and the eng write policy named ~/.gstack/projects/ and
~/.gstack/analytics/, while the writers (tasks-section.ts, testing.ts,
spec-review.ts) use $GSTACK_STATE_ROOT. When the state root is relocated the
prose pointed at the wrong directory.

W7a pin: test/plan-review-cases.test.ts:852 pinned the stale path; it now
checks the discovery path is under $GSTACK_STATE_ROOT and that the hardcoded
~/.gstack/projects/{slug}/ is gone. expected red until the regeneration commit

* plan-ceo-review: remove the handoff-note reader and cleanup for files nothing writes (s04 F6, s04 F7)

CEO handoff notes (*-ceo-handoff-*.md) have not been written since v0.11.5.1;
no template, resolver or bin script produces them. The reader cost a bash
call on every CEO run, depended on $SLUG/$BRANCH from another shell, and
looked under $GSTACK_STATE_ROOT where pre-state-root notes never lived.

W7a pin: test/skill-ceo-section-ordering.test.ts:320 sliced up to the
removed '## Handoff Note Cleanup' heading; it now slices to the next heading
('## Review Log') with the same assertions.
Note: the ceo-mode-option / ceo-mode-prerequisite PTY cases fail under the
image's Bun 1.3.14 with or without this change and pass under Bun 1.4.0.
expected red until the regeneration commit

* plan reviews: replace context-pressure triage with the full-completion rule (s04 F1; manual merge of s05 M2)

plan-ceo-review, plan-design-review and plan-devex-review ranked sections
'under context pressure' ending in 'Everything else', which licensed dropping
sections their own anti-skip rules require. plan-eng-review already replaced
the same block with 'Complete every required stage ... do not preemptively
warn'. All three now use that rule and name what may never be shortened
(design: Step 0, Step 0.5 mockups unless the user opted out, Passes 1-7;
devex: Step 0 with persona/empathy/benchmark/magical moment/TTHW, Passes 1-8).

s04 F1 noted the same block in the s05 files; the design copy is a manual
merge of s05 M2 (drops 'non-negotiable', which contradicted the user's
explicit 'skip mockups' opt-out). No test pins the removed text.
expected red until the regeneration commit

* plan-ceo-review: shared-code guidance matches the eng review (s04 F4, s04 F5)

'DRY: flag repetition aggressively' and 'DRY violations. Be aggressive.'
contradicted plan-eng-review's newer rule (require common behavior and
improved reliability or net savings; similar-looking code alone is
insufficient), so the two reviews gave the same plan opposite advice. Both
CEO lines now follow the eng rule; Section 5 keeps the file:line citation.
No test pins the removed text. expected red until the regeneration commit

* plan reviews: normal-volume rules; eng sections report every finding (s04 F8, s04 F10, s04 F11)

- F8: eng 'at most 8 findings' per section conflicted with 'never condense or
  skip' and left a 9th high-confidence finding nowhere to go; report every
  main-report finding, most severe first (confidence calibration still
  suppresses low-confidence ones).
- F10: drop 'It is not optional.' (the anti-skip rule already covers Section 2).
- F11: 'ALWAYS a smell' -> 'a smell'.
No test pins the removed text. Two eng PTY cases fail under Bun 1.3.14 and
pass under Bun 1.4.0 (CI pin). expected red until the regeneration commit

* plan reviews: '## CRITICAL RULE — How to ask questions' -> '## How to ask questions' (s04 F9, s05 L3)

The bodies are plain procedure; the caps label added nothing and competed
with the CRITICAL GAP output label. Applied in plan-ceo-review,
plan-design-review and plan-devex-review sections (s05 L3 promoted by the
plan).

W7a: test/devex-finding-fixture.test.ts:83-84 now finds the section by a
heading regex ending in 'How to ask questions'; the skill-e2e-plan-ceo-plan-mode
failure diagnosis names the new heading.
Judge anchors NOT changed here (builder E moves them to structural anchors):
skill-ceo-section-ordering.test.ts (old :938), workflow-excerpt.test.ts:109,296,
skill-llm-eval.test.ts:638, workflow-judge-input.test.ts:366,373; the mirror
in touchfiles.test.ts:219 also uses '## CRITICAL RULE'. These stay red until
E's anchor commit lands. expected red until the regeneration commit

* autoplan: phase drivers match the loaded review skills (s05 H1, s05 H2, s05 H3, s05 H4, s05 H5, s05 M5)

- H1/H2: the skip list names the current headings ('Step 0: Detect platform
  and base branch', 'Design Outside Voices (independent)'); the old names
  never matched, risking a duplicate base-branch detection or outside-voice pass.
- H3: the DX journey map is the 6-stage table from Step 0F, not 9 stages.
- H4: the DX checklist names Step 0 'DX Investigation' and scores only after
  the evidence exists, as plan-devex-review requires.
- H5 (manual merge): the eng test-plan artifact uses the eng skill's
  {user}-{branch}-eng-review-test-plan-{datetime}.md name under
  $GSTACK_STATE_ROOT/projects/$SLUG (consistent with s04 F3).
- M5: the Decision Audit Trail separator has 7 columns to match its header.
No test pins the removed text. Three autoplan PTY cases fail under Bun 1.3.14
and pass under Bun 1.4.0. expected red until the regeneration commit

* plan-design-review: scope gate runs Step 0 before mockups (W5.4, s05 H6)

The scope gate said 'audit → mockups → Step 0' while the body runs Step 0
(0A-0D) first and 0D announces 'I'll generate visual mockups next'. Per the
W5.4 decision, the three gate phrases now read audit → Step 0 → Step 0.5
mockups. 'Reply with A, B, or C' is unchanged.

W7a pins:
- test/plan-scope-selection.test.ts:616 pinned the old order sentence; it now
  checks the order of the 'After resolution:' steps (preamble, base branch,
  audit, Step 0, mockups). Negative control: the old order turns it red.
- test/gen-skill-docs.test.ts:4737 normalizer strips the design-only
  'Then run the pre-review audit, ... against that plan.' sentence by pattern.
expected red until the regeneration commit

* plan-design-review: mockup default and retrospective at normal volume (s05 M1, s05 M3, s05 M4)

- M1: 'YOUR PRIMARY TOOL' / 'The rule is simple ... Don't ask permission ...
  Mockups ARE the plan' -> generate mockups without asking when the plan has
  UI and the designer is available, with the reason; skip only under the
  Step 0.5 conditions (which include the user's opt-out).
- M3: 'be MORE aggressive' on previously flagged areas -> check whether the
  earlier issue recurs or was only partly fixed and say so.
- M4: drop the unverifiable 'Tier 2+ (15+ RPM)' rate-tier aside; keep the
  one-at-a-time rule and the /design-shotgun pointer.
(s05 M2 landed with s04 F1.) No test pins the removed text; the remaining
failing case is the plan-scope-selection PTY case that fails under Bun 1.3.14
and passes under Bun 1.4.0. expected red until the regeneration commit

* plan-devex-review: DX TRIAGE keeps every pass and reports blockers only (W5.5, s05 H7)

The Mode Quick Reference let TRIAGE skip the benchmark, the magical moment
and passes 2/4-8, contradicting the anti-skip rule, the 0C target gate and
the 'executed all 8 DX passes' self-check. Per the W5.5 decision the TRIAGE
cells now read 'Target only' / 'Verify exists' / 'All 8, blockers only',
matching the Rating Method's 'flag adoption blockers; skip nice-to-haves'.
No test pins the removed cells. expected red until the regeneration commit

* ship, review, land-and-deploy, landing-report: fix stale cross-references (s06 H1, s06 H2, s06 H3, s06 H4, s06 H5, s06 H8, s06 M6)

- H1: TODOS-format cites /ship Step 14, not 5.5.
- H2: the land readiness review is the checklist /ship applies in Step 9 (there
  is no Step 3.5).
- H3: the Apple adapter names the no-credential-talk rule by location instead
  of 'line 14' (it renders at line 16).
- H4: greptile triage diffs against origin/<base> from the caller's detected
  base, not origin/main.
- H5: the land Important Rules recap matches the steps (4-minute CI rounds in
  Step 3; 30 seconds for merge readback and deploy status).
- H8: landing-report calls the util once per bump level (four), and drops the
  unsupported 'cached by bun' claim.
- M6: landing-report says what its base detection does instead of claiming
  it matches other skills.
No test pins the removed text. expected red until the regeneration commit

* fix(land-and-deploy): readiness gate uses the project's test command and the live eval store (s06 H6, s06 H7)

- H6 (manual merge): TEST_COMMAND comes from CLAUDE.md or AGENTS.md; if
  neither documents one, ask the user instead of defaulting to 'bun test'
  (wrong for non-Bun projects and a bare suite run in gstack). The audit
  hunk's 'persist the answer to CLAUDE.md' write is not adopted: it adds a
  write the readiness gate never had.
- H7: E2E and judge results are searched in the eval store the collector
  writes (~/.gstack/projects/<slug>/evals/ and its shards/*/ directories),
  then the legacy ~/.gstack-dev/evals/ fallback; the gate no longer reports
  'no E2E tests run today' when results exist.

test/state-root-ratchet.allowlist.json: allowlist the EVAL_DIR line with its
reason (test/helpers/eval-store.ts writes under os.homedir()/.gstack, so the
reader must use the same path, not $GSTACK_STATE_ROOT).
Parity cap (test/helpers/carve-guards.ts): land-and-deploy maxSizeRatio
1.10 -> 1.108 (measured 1.1076 at the branch tip, including the W7f marker).
expected red until the regeneration commit

* ship: Step 9 Pass 1 defers to the checklist's critical categories (W5.3, s06 H9)

/ship restated the checklist's Pass 1 as two of its five CRITICAL categories
(missing Race Conditions, Shell Injection, Enum Completeness). Per the W5.3
decision it now names 'the checklist's Pass 1 categories', matching /review.
No test pins the removed text. expected red until the regeneration commit

* land-and-deploy: host-neutral post-merge branch example (s06 M5)

The squash-merge cleanup example created 'codex/post-merge-pr-N' branches on
every host. The example is now 'post-merge/pr-N', naming Codex Desktop's
codex/ prefix as the host-specific case and keeping the detached-HEAD reason
and the no-force-push rule. No test pins the removed text.
expected red until the regeneration commit

* ship: drop issue/incident IDs and the 'continue silently' suppressor (s06 M2, s06 M3, s06 M4, s06 M7)

- M2/M3: '(#1946)' and '(#1079)' removed from the credential guard and REST
  fallback headings; the rules are unchanged.
- M4: the audit-failure fallback states the reason (a skipped audit lets
  unverified plan items ship as done) instead of 'the failure shape that
  VAS-449 surfaced'.
- M7: 'Continue silently — just note the counts briefly' -> report the pass
  counts in one line and continue to Step 6 (Step 16 cites these counts).

Not changed: s06 M1, the Step 16 IRON LAW (UC2 gate keeps its wording).
W7a pin: test/ship-plan-completion-invariants.test.ts:127 pinned the VAS-449
sentence; it now checks that the audit-failure fallback still forbids a
silent fail-open (case-insensitive). Negative control: deleting that
sentence turns it red. expected red until the regeneration commit

* test: anchor plan-design excerpts on the question-procedure heading, not '## CRITICAL RULE' (W7a judge-anchor move; workflow-excerpt.test.ts:109,296)

The end marker now accepts a RegExp. ASK_QUESTIONS_HEADING matches the heading
with or without its 'CRITICAL RULE — ' prefix, so the excerpt is byte-identical
on the current templates and survives the s04 F9 / s05 L3 rename.

* test: plan-design judge-input case anchors on the question-procedure heading (W7a judge-anchor move; workflow-judge-input.test.ts:366,373)

* test: plan-design workflow judge ends its slice at the question-procedure heading (W7a judge-anchor move; skill-llm-eval.test.ts:638)

Judge input is byte-identical on the current templates; judge prompts and
schemas are unchanged.

* test: CEO governing-stage order anchors on the question-procedure heading (W7a judge-anchor move; skill-ceo-section-ordering.test.ts:938)

* test: dedicated-tools overlay gates on correct output plus 20% fewer Bash calls (Eng W7 overlay predicate; replaces overlay-nudges.ts:213 taskCorrect)

Every ON trial must pass the fixture's verify (already a valid-measurement
requirement) and lowerIsBetter20Pct must hold; when the OFF median is 0, every
ON trial must also be 0. assessOverlayArms enforces both through a new
fixture.gate. Overlay contract bumps to v3 so v2 records are never
reinterpreted. overlay-measurement gains rows for correct+worse (fails),
20%+incorrect (fails) and the zero baseline (passes only at 0). The prose
comment follows the s01 M1 claude.md wording.

* test: document-release E2E stops coaching the rules it measures (W7b; UC2 gate eval for NEVER CLOBBER / NEVER BUMP)

Drops the IMPORTANT block at skill-e2e-workflow.test.ts:76-83, including
'auto-approve everything' and the CHANGELOG/VERSION instructions; keeps only
the environment facts (non-interactive, no remote). Asserts the outcome: the
original CHANGELOG entries remain, VERSION still reads 1.1.1, no commit after
the fixture head touches VERSION, and README mentions Feature C.

* test: routing E2E uses the routing instruction gstack ships (W7b routing fixture; skill-routing-e2e.test.ts:113)

readShippedSkillRouting() parses the '## Skill routing' section from the
routing-injection heredoc in bin/gstack-skill-start at test time. The journey
fixture now carries that instruction instead of 'ALWAYS invoke … FIRST action.
Do NOT answer directly', and still withholds the per-skill rule list so a
regressed description can fail. bin/gstack-skill-start and the helper join the
journey, context-skills and skillify touchfiles. A free check catches heredoc
drift. Runs on unchanged templates set the new baseline.

* test: context-skills E2E routes with the shipped ## Skill routing section (W7b routing fixture)

Replaces the shouted 'ALWAYS invoke it … FIRST action' fixture and its
hand-written context-save/restore rules with the section gstack appends to
CLAUDE.md, which already routes save/resume to those skills.

* test: skillify E2E stops scripting the skill and routes with the shipped section (W7b skillify prompt + routing fixture; W7d skillify narration ban)

- CLAUDE.md fixture: the shipped ## Skill routing section replaces 'ALWAYS
  invoke it via the Skill tool as your FIRST action' and its hand-written rules.
- scrape-match-path: prompt is 'Run /scrape latest hacker news stories'; the
  'You MUST follow … 1. Run skill list … Do NOT enter the prototype phase'
  script is gone. Asserts the outcome: the bundled skill ran, no goto
  prototyping, no new skill or skillify staging written. Listing is logged.
- scrape-prototype-path: drops the five-step script and the self-fulfilling
  'Suggest /skillify'; keeps the user's JSON shape request.
- skillify-provenance-refusal: prompt no longer dictates the refusal.
- Narration grader moves to helpers/skill-body-narration.ts: line-start
  'I '/'I'll '/'Let me ' plus mid-sentence 'let me <verb>', so 'let me know'
  is legal. Stored known-good/known-bad bodies show it still rejects every
  known-bad case the old /Let me /i ban caught.

* test: land-and-deploy E2Es state environment facts instead of scripting the skill (W7b deploy prompts)

The workflow, first-run and review-gate prompts drop the 'IMPORTANT … simulate:
1. Detect … 2. Infer … 3. Note …' scripts and the answers they supplied
(platform, app, URL, merge method, 'Eng Review should be NOT RUN'). They keep
the facts the fixture cannot provide (no remote, no PR, gh/fly unavailable,
gstack-review-read output, report directory). Assertions now check the outcome:
the artifacts name Fly and test-app, and the readiness report records the
missing review. Canary, benchmark and setup-deploy prompts are unchanged.

* test: ship triage asserts each failure's in-branch/pre-existing label (W7b ship triage)

The old check passed if the words 'in-branch', 'pre-existing', 'math' and
'string' appeared anywhere in the final message, in any pairing. The prompt
now asks for a closing JSON map; helpers/ship-triage-labels.ts reads it (or
unambiguous per-failure lines) and the test requires truncate/string ->
in-branch and divide/math -> pre-existing, plus a Bash call that ran the
suite. Stored outputs show swapped, words-only and one-label runs fail.
Drops the untyped recordE2E extras (typecheck baseline shrinks by one).

* test: design-review-fix asserts a fixed design issue, not only the Aside ritual (W7b design-review-fix)

Keeps the engine-routing checks (Aside driven, retired binary unused) and adds
the outcome: design-audit.md exists, names a seeded defect, and index.html or
style.css differs from the initial fixture commit. The style(design) commit
prefix is not required. This case still cannot gate the design-review
stop-condition rewrite (it self-skips without Aside and caps fixes in its
prompt), per the Eng item.

* test: CEO HOLD preservation grader checks the decision, not its phrasing (W7d CEO mode-routing regex judge, ceo-mode-option.ts:410-440)

Drops the exact 'Keep <x> in scope' / 'Defer <x> to TODOS' labels, the
'D<n> — Keep … in scope, or defer' title, the '<plan>.md lists' phrase and the
2-of-5 keyword quota. Keeps the evidence bindings (native answer, session,
timestamps, source read, mode context, later withdrawal) and now requires: one
keep option and one defer option, the answer selects keep, a word of the kept
label names a listed plan requirement, no expansion language, and an ELI10 +
Stakes rationale of at least 20 words (so a bare mode label still fails).
Stored captures: both retained known-good runs pass; every existing known-bad
mutation still fails; four rewordings the old grader rejected now pass; three
new known-bad cases fail. The Net/Note single-brief checks (t02 #3) are
unchanged: the 'second sentence after Net' row adds scope ('Also add shared
views'), so it stays a known-bad.

* test: docsync observer accepts harmless read wrappers (W7d docsync observer)

docsCommandAllowed now unwraps, on read-only commands only (pwd, ls, cat,
stat, sha256sum, Git reads): a leading 'cd <fixture repo> &&', '2>/dev/null',
a trailing '|| true', '--no-pager', '-C <fixture repo>' and '-c' with
core.pager=cat / core.quotepath / color.* settings. Lifecycle and script calls
stay literal so their failures cannot be suppressed. Writes, other repos,
--git-dir/--work-tree, config that can execute code, real composition and
redirects stay rejected.

Stored commands (fixtures/docsync-observer-commands.json): 11 known-good read
forms, which the old observer rejected, now pass; all 17 known-bad forms still
fail under the new and old observers. docsync-command-grammar moves its three
harmless rows to an accept test and keeps unsafe variants of them as rejects.

* test: AUQ matrix fails only on the format fields software reads (W7d AUQ matrix emoji check)

auqMachineFormatProblems() fails a first-question run when the
Recommendation: line is missing or empty, or when the brief does not label
exactly one option (recommended); recommendation substance >= 4 is unchanged.
ELI10, Pros / cons, ✅/❌ and Net: are still scored and logged but no longer
fail a paid case. scoreAuqFormat and the consistency suite are unchanged.
Stored briefs: three known-good (including one without emoji or Net) pass;
four known-bad (no or empty Recommendation, zero or two (recommended)) fail.

* test: coverage audit grades a closing tested/untested summary before the diagram grammar (W7d coverage-audit-evidence)

The diagram grammar grew through four waves of capture patches to accept
correct-but-different diagrams. The E2E prompt now asks for one closing JSON
line {"tested": [...], "untested": [...]}; coverageAuditVerdict passes the
outcome when processPayment is only in tested and refundPayment only in
untested (last summary wins; quoted or example summaries ignored). The legacy
grammar stays as a fallback, so every stored capture keeps its verdict
(all existing rows unchanged). Read evidence and capture success are still
required. Stored summaries: three known-good (which the old grader rejected)
pass; seven known-bad fail.

* test: disabled outside-plan grader reads the run's own coverage self-report (W7d disabled-plan-review-evidence)

The false-completion oracle kept being patched to tell historical review-log
mentions from current claims. The E2E prompt now asks for a closing JSON line
{"outside_review_this_run": ...}. When present, the coverage claim is that
report ('completed' fails) plus any undenied 'both reviewers agree';
'disabled' also satisfies attribution. Runs without it keep the existing
prose oracle, so all stored captures and mutation rows keep their verdicts.
Dispatch, CLI execution, preflight and persisted-disabled checks are
unchanged. Stored outputs: three known-good pass (the old grader rejected
the table form); four known-bad fail.

* test: investigate boundary grader checks the verified fix, not one Edit and one verify (W7d investigate exactly-one ritual)

Completion now needs at least one allowed Edit, the correction in place, at
least one verifier run whose final status is 0, and no edit after the final
verification. The ending-error case still requires a single verifier run
(status 69), because that fixture's error tells the actor to stop without
retrying. Ownership acquisition/release checks are unchanged. New free
protocol controls: a passing re-verification passes (the old grader failed
it); a failing final re-verification and a retry after the ending error both
fail, as they did before.

* test: CEO split-overflow call floor is two, the minimum for five options at four per call (W7d split-overflow 4-call floor)

The regression is a candidate silently dropped to fit the four-option cap.
The scope validator already requires an offered Include/Defer/Cut disposition
for all five candidates; the extra N-1 call floor failed agents that batch
compatible candidates, which the test header calls legitimate. FLOOR now comes
from CEO_SPLIT_CALL_FLOOR = 2. Stored shapes (ceo-split-floor.test.ts): five
sequential, 2-call and 3-call batches pass; a dropped candidate fails with
'missing target decisions'. The registration controls keep their verdicts;
the 3-call plan_ready scenario still fails, now on missing decisions.

* test: shared-libs PR coverage checks the stated page budget, not per-PR ceilings from one run (W7d shared-libs per-endpoint page caps)

prCoverageRequestViolations keeps the skill's stated older-open-PR budget
(five metadata pages, fifty file-list pages plus recent PR 7) and the coverage
outcome (PR 42's file list read from page 1 through page 2). It drops the PR 42
three/four-page and single-first-page ceilings, which were tuned to one
observed capture with a truncated first view. The judge criteria are
unchanged. Stored endpoint lists: three known-good (including a re-read first
page the old ceilings rejected) pass; four known-bad plus an over-budget
file-list run fail. incompleteFirstFileView is no longer used by the paid case.

* test: test-value review accepts any caller search, not only the literal 'git grep' (W7d literal git grep in test-value)

recordsNonTestCallerSearch keeps the value-card contract (non_test_callers
recorded) and accepts git grep, rg, grep or the Grep tool as the search.
Severity checks and the golden exclusion are unchanged. Stored evidence:
four known-good (three the old check rejected) pass; three known-bad (no
search, no caller count, prose only) fail.

* test: plan-format accepts any statement that kind-differentiated options are not comparable on coverage (W7d 'options differ in kind' phrase)

The outcome, no fabricated Completeness: N/10, stays asserted. The required
note now matches any brief wording of the same fact (KIND_NOTE_RE in
helpers/plan-format-kind-note.ts). The format-compliance suite that pins the
declared kind-note string is unchanged. Stored lines: four known-good pass,
three known-bad (a completeness score, a recommendation, 'kind to the user')
fail.

* test: plan-mode smokes check the targeted plan, not the scope-gate announcement, and reject plan-file ## Decisions (W7d exact scope-gate line; W7g plan-mode ## Decisions)

- Seeded eng/design plan-mode cases keep 'no scope menu' and replace the
  exact 'Scope gate: plan mode — auto-selected B' requirement with
  seededPlanTargeted(): the announcement or native seeded-plan selection, or
  seed-only plan names appearing in the review output after the slash command
  (trackTokens). The announcement is logged.
- The AskUserQuestion format forbids writing a decision to the plan file as a
  substitute for asking. assertNoPlanFileDecisions() fails any plan_ready run
  whose plan carries ## Decisions (eng, design, devex, office-hours
  auto-mode). The AUQ-blocked cases (eng seeded, devex, office-hours) now
  require outcome 'asked' (AskUserQuestion variant or prose fallback).
- plan-mode-evidence registration controls keep their verdicts; the
  missing-decisions control now fails on 'without a question'.
- plan-mode-targeting.test.ts stores known-good (announcement; paraphrase plus
  seed content) and known-bad (neither; an untracked token) observations and
  plan files with and without ## Decisions.

* test: plan-design judge selection check anchors on the question-procedure heading (W7a judge-anchor move; touchfiles.test.ts:219)

Same structural anchor as the other judge-anchor moves; selects the same text
on the current templates and after the '## How to ask questions' rename.

* test: CEO plan-mode failure message names the current rule, not the '## CRITICAL RULE' heading (t13 F20)

* test: browse reference judge gates on its stated floors; the baseline comparison is recorded, not a ratchet (W7d browse judge baseline ratchet; t08 F7)

expect(regressions).toEqual([]) compared a noisy three-sample mean against
eval-baselines.json (clarity 4), silently raising the stated clarity floor of
3. The comparison stays in judge_reasoning; pass and assertions use
BROWSE_JUDGE_FLOORS (clarity 3, completeness 4, actionability 4). Judge
prompt and schema are unchanged. Stored panel means: three known-good (two the
ratchet rejected) pass; three below-floor panels fail.

* test: eval-kinds drops the exact 22 behavior / 25 judge census (W7d exact count in eval-kinds; t12 F6)

The kind contract is already asserted: every live case has one kind, kinds
are rule/behavior/judge, every LLM judge is judge-kind, every behavior case
has a literal registration and a BEHAVIOR_WHY, and the named contract-shaped
cases stay rule. The census count only broke on adding or retiring a case.

* test: plan-eng artifact case binds to the hermetic state root and requires the artifact; report case drops its shouted requirement (W7g plan-artifact fixture binding; W7g skill-e2e-plan.test.ts:479,720)

- plan-eng-review-artifact: actor, observer and cleanup use one isolated
  GSTACK_HOME (the hermetic run's, or a seeded temporary one when hermetic
  mode is off) and the slug gstack-slug resolves for the fixture, instead of
  os.homedir()/.gstack/projects/test-project. A guard refuses a state root
  under the real home, and cleanup removes only the fixture's project
  directory. The 'IMPORTANT … you MUST write the test-plan artifact' line is
  replaced by the environment fact (state root and slug). The soft warning
  becomes an assertion: exactly one new *eng-review-test-plan-*.md about the
  reviewed dashboard change.
- plan-review-report: removes 'CRITICAL REQUIREMENT … MOST IMPORTANT
  deliverable' and the restated report contract; keeps 'plan.md is the plan
  file'. The existing assertions (report present, last, rows, unresolved
  status) now measure the skill's own report prose.

* test: session-runner-tools checks harness prompt properties, not a byte-for-byte copy (W7e session-runner-tools.test.ts:544; t09 F7, F8)

The ship capture prompt test asserts its purpose (no CEO-only writing
guidance) plus the functional rules: one allowed skill path, AskUserQuestion
unavailable, the section Read rule, no git/gh/mutation, and the REPORT.md
path. The two 'MUST actually Read' pins match the Read rule case-insensitively.
Negative control: dropping the no-mutation rule from the harness prompt turns
the test red.

* test: section-capture native tools checks harness prompt properties, not a copied golden (W7e section-capture-native-tools.test.ts:467; t04 F4)

Asserts the absolute skill path is the only SKILL.md named, the scenario,
AskUserQuestion unavailable, the section Read rule (case-insensitive, also at
:308), no git/gh/mutation, and the REPORT.md path. argv/tool-list checks are
unchanged. Negative control: dropping the no-mutation rule turns it red.

* test: office-hours review-preservation judge test keeps its schema and untrusted-data checks, not its wording (W7e office-hours-completion.test.ts:416-429; t09 F10)

Drops 13 phrase pins on the judge prompt. Kept: both inputs carried as JSON,
the response schema (wire format), and the separate untrusted-DATA test. The
validator behavior tests already use stub judges. Negative control: changing
the schema string in the prompt builder turns the test red.

* test: docsync lifecycle interface drops pins on its own helper prompt (W7e docsync-lifecycle-interface.test.ts; t02 #17)

docsNativeInterface is the test helper's own prompt, so its sentence pins
only checked that the helper says what it says. Kept: owned skill path, the
two extracted lifecycle commands (exact), the scripts-list interpolation, the
repo path and Git read examples, and every docsCommandAllowed /
docsToolFailures behavior check that enforces the interface.

* test: shared-libs revalidation prompt keeps its machine-shaped checks, not ~69 phrase pins (W7e shared-libs-revalidation-prompt; t11 #3, #8; t05 F17)

reviewRevalidationPrompt and reviewPrompt are test-owned harness prompts
(non-test callers: 0); the paid shared-libs E2Es measure whether they work.
Kept: the three exact receipt commands, base-prompt prefix and resumed suffix
identity, branch selection (actor / default / resumed coverage), the state
path and turn-budget interpolations, read-record-before-diff and
first-response-before-start ordering, no embedded token, no supplied Skip
answer, reusable:true before reusable:false, the captured R59 stage boundaries
in the exclusion list, asset roots and interfaces, and the stale step-6
reference ban. The review template's Step 4.7/4.8 completion rule is matched
by step number on one line, not by sentence. Negative control: reordering the
diff-read instruction turns the test red.

* test: plan-design SDK fixture pins its harness prompt once, by ordered properties (W7e plan-design-sdk-fixture.test.ts:91-120; t01 F13)

The prompt was pinned three ways (sha256, ~20 sentences, an order check).
Kept: the skill and lazy-section paths, the plan path, the ordered
read -> review -> edit -> Write -> read-back -> summary steps, and the
no-browsing rule (case-insensitive). Dropped the hash and sentence pins.
Negative control: removing the read-back step turns it red.

* docs: overlay benchmark contract v3 (dedicated-tools gate = correct output + 20% fewer Bash)

* test: dedicated-tools overlay case pinned to claude-opus-5-5 (E2)

* chore: regenerate SKILL.md, goldens and context-budget fixture for the Opus 5.5 prompt cleanup

* document-release: CHANGELOG-preservation and VERSION-ask rules at normal volume (s10 M4, M5; UC2, eval-gated)

* chore: regenerate document-release

* Revert "test: dedicated-tools overlay case pinned to claude-opus-5-5 (E2)"

This reverts commit 737bf478d9aefeb8c333f078b8a16877dcd8f9a7.

* test: memory-pipeline E2E seeds transcript consent (W4 consent-first default)

* test: plan-review-report free check follows the de-coached prompt (W7g)

* test(W7h): drop issue-number pins in land-and-deploy-postfail, match guards case-insensitively

W7h (t09 F12, F13). One test file.

Removed assertion groups:
- `cli/cli#3442` / `cli/cli#13380` pins (t09 F12). Invariant: none of its
  own (upstream archaeology; the IDs stay in the template, only the pin goes).
  The behavior they decorated, "after any non-zero `gh pr merge` exit, query
  authoritative state before retrying", is now a kept free meaning check.
- `Universal invariant` label pin. Invariant: query-before-retry. Remaining
  protection: the new case-insensitive check
  `/non-zero exit from `gh pr merge`, query authoritative PR state before retrying/i`
  (stronger: the mutation that inverts the rule now goes red; the old test
  stayed green).

Loosened to case-insensitive meaning checks (kept, W7 keep list, data-loss
and destructive-op guards): STOP worktree cleanup without removing, no
`--force`, primary working tree never removed, CLOSED -> stop, never replay a
merge after MERGED, one direct fallback, no fallback from a direct attempt,
the OPEN/no-auto/no-queue fallback precondition, branch-absent vs check-failed
messages. Gate-tier E2E also covering the flow: land-and-deploy-workflow
(gate), judge `land-and-deploy/SKILL.md workflow` (gate lane).

Cosmetic pins dropped (no control needed): capitals on STOP / Do NOT.

Mutation log: .context/mutation-land-and-deploy-postfail.md (12 mutations,
0 unexpected: reorder, field delete and 7 safety deletes red; 3 rewordings green).

* test(W7c): ship-apple-gate matches its gates on meaning, not capitals

W7c (t06 #19). One test file.

Removed assertion groups:
- "A genuine blocker may require a safety or named documentation-risk
  decision" and "routine interactions and blocking decisions above" sentence
  pins. Invariant: routine release authorization never waives a blocking
  safety/docs decision. Remaining protection: kept free meaning check
  `/stop for that decision rather than treating release authorization as a waiver/i`
  plus `/named documentation-risk exception before distribution/i`.
- Full-sentence "in `read-only` mode against the selected release source".
  Invariant: the docs preflight is read-only. Remaining protection: kept free
  check on "`read-only` mode".

Loosened (case-insensitive, kept): apply-only-to-repository-landing scope
line, CLASSIFY-before-credentials (credential boundary), never abort a store
release over branch topology (demonstrated failure), two routine interactions
(numeric limit), stop-for-decision (consent/stop line).

Kept unchanged: manifest trigger routing, Apple read before the branch gate,
byte-exact branch gate appearing once, docs preflight before the
authorization moment, fastlane/iris/appPriceSchedules command anchors,
stale-wording bans. No gate-tier E2E covers the Apple adapter, so every
invariant keeps a free check here.

Cosmetic pins dropped: capitals on ONLY / CLASSIFY / STOP.

Mutation log: .context/mutation-ship-apple-gate.md (14 mutations, 0
unexpected: 2 reorders, 1 marker and 6 safety mutations red; 5 rewordings green).

* test(W7c): ship-reentry-gates keeps stop/consent meaning, drops sentence pins

W7c (t08 F3). One test file.

Removed assertion groups (invariant -> remaining protection):
- "Retain queued fixes and restore coverage" -> a missing reviewer keeps
  queued fixes: gate E2E ship-skipped-queued-finding (gate) and judge
  `ship/SKILL.md workflow` (gate lane); the stop itself stays a free check.
- qa decline print line ("Test bootstrap previously declined — skipping.")
  -> qa skips bootstrap after a saved decline: kept free meaning check
  `/BOOTSTRAP_DECLINED ... skip the rest of bootstrap/i`…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant