Skip to content

feat(planning): route the ungrillable question to a prototype detour, and leave plan mode off while interviewing - #3045

Merged
kyle-sexton merged 5 commits into
mainfrom
claude/pocock-steering-course-00zkvd
Aug 20, 2026
Merged

feat(planning): route the ungrillable question to a prototype detour, and leave plan mode off while interviewing#3045
kyle-sexton merged 5 commits into
mainfrom
claude/pocock-steering-course-00zkvd

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Closes #2998

Summary

Course lane 5's /planning:audit-answers pass — two fresh-context validators reading the upstream
primaries independently — overturned the lane's provisional zero-work-items answer and surfaced two
grilling-family techniques absent from planning:interview. This adopts both, and reconciles the
planning:plan sentence that sat in tension with the second.

Documentation only; no behavior change. planning 0.32.0 → 0.33.0.

Fix

The ungrillable question → prototype detour. Some questions cannot be answered by asking,
however well the round is framed: the user has to see a thing and react to it before they know what
they think. Upstream fires the escape mid-grilling ("stop grilling… build the throwaway version…
come back and answer in one line"). Our route existed only downstreamwayfind, plan,
brainstorm — while the interview's mid-session composition named research, exploration, and
handoff only, and its categorization taxonomy had no needs-an-artifact arm.

  • interview/SKILL.md "Mid-interview composition" now carries the detour, routing a look-and-feel
    question to /prototype:explore-directions and a logic/state/data-shape question to
    /prototype:pressure-test (invoked via the Skill tool, if installed), then returning to answer in
    one line. A matching composition-table row keeps the two surfaces consistent.
  • interview/context/loop.md gains a Needs-an-artifact arm beside resolvable / blocked /
    defer-with-assumption / defer-fully, marked explicitly as a route, not a deferral.

The prototype is the instrument that produces the answer, not a deliverable — it is thrown away. The
question is registered open before the detour so it cannot be lost across it.

Plan mode off while interviewing. Beyond upstream's taste point, the audit surfaced a mechanical
edge: the ask-time open-question register is a disk write (the ledger's ## Open-question register section — verified against context/loop.md "Write at ask-time, not at answer-time", not
inferred), and plan mode's read-only enforcement blocks it. A round asked under plan mode therefore
leaves nothing on disk holding it — precisely the failure the register exists to prevent,
reintroduced by the permission mode.

  • New interview/context/gotchas.md entry under the Q&A surface, beside the other register gotchas.
  • plan/SKILL.md "Plan Mode Integration" reconciled with lane 4's asset-rush doctrine: its
    plan-mode round is now scoped to a scoping confirm, and substantive what are we building
    questions route to /planning:interview outside plan mode, on both the register-write mechanics
    and the doctrine that plan mode primes the run toward producing the asset while the job is still
    reaching shared understanding.

Merge with main (2026-08-20)

main shipped planning 0.32.0 for #2936 while this branch had already claimed that number for
#2998, so the branch merged main and re-numbered:

Verification

Full local gate re-run on the merge commit, all green:

Gate Result
scripts/validate-plugins.sh all manifests + catalog validated
generate-catalog.mjs --check / generate-cheatsheet.mjs --check both in sync
check-changelog-parity.sh --check / --check-order / --check-bump origin/main / --check-preserved origin/main all pass; 83 changelogs newest-first, 55 headings preserved
check-changed-skills.sh origin/main 2 skills checked, 0 failed
markdownlint-cli2 (6 touched files) 0 issues
typos clean
planning test suites (standards-binding, goal-condition-length, check-open-questions) PASS=9 FAIL=0, plus both script suites

The three check-changed-skills warnings are pre-existing on origin/main, not introduced here:
the SKILL.md soft line-count target (interview and plan were both already over the 200 target before
this change) and the numbered-procedure completion-criteria warning on the unchanged
Plan Review Mode list. The REcompile() - panic lines in that run are the known mawk/ERE-interval
artifact tracked as #3005, not a failure — the script exits 0.

Related


🤖 Generated with Claude Code

https://claude.ai/code/session_01QbfCrj3X9FfGL7VRZYmrn4


Generated by Claude Code

…2998)

Course lane 5's `/planning:audit-answers` pass (two fresh-context
validators against the upstream primaries) overturned the lane's
provisional zero-work-items answer and surfaced two grilling-family
techniques absent from `planning:interview`.

## The ungrillable question

Some questions cannot be answered by asking, however well the round is
framed: the user has to see a thing and react to it. Upstream fires the
escape mid-grilling — stop grilling, build the throwaway, come back and
answer in one line. Our route existed only downstream (`wayfind`,
`plan`, `brainstorm`); the interview's mid-session composition named
research, exploration, and handoff only, and its categorization taxonomy
had no needs-an-artifact arm.

- `interview/SKILL.md` "Mid-interview composition" now carries the
  detour, routing a look-and-feel question to
  `/prototype:explore-directions` and a logic/state/data-shape question
  to `/prototype:pressure-test` (via the Skill tool, if installed), with
  a matching composition-table row.
- `interview/context/loop.md` gains a **Needs-an-artifact** arm beside
  resolvable / blocked / defer, marked explicitly as a route rather than
  a deferral.

The prototype is the instrument that produces the answer, not a
deliverable. The question is registered `open` before the detour so it
cannot be lost.

## Plan mode off while interviewing

Beyond upstream's taste point, a mechanical edge: the ask-time
open-question register is a disk write (the ledger's `## Open-question
register` section), and plan mode's read-only enforcement blocks it — so
a round asked under plan mode leaves nothing on disk holding it, the
exact failure the register exists to prevent.

- New `interview/context/gotchas.md` entry under the Q&A surface, beside
  the other register gotchas.
- `plan/SKILL.md` "Plan Mode Integration" reconciled with lane 4's
  asset-rush doctrine: its plan-mode round is scoped to a scoping
  confirm, and substantive what-are-we-building questions route to
  `/planning:interview` outside plan mode.

## Verification

Documentation only; no behavior change. planning 0.31.0 -> 0.32.0.
validate-plugins, both generator --check runs, all four
check-changelog-parity modes, check-changed-skills (2 checked, 0
failed), markdownlint on the five touched files, typos, and the three
planning test suites all pass. The two check-changed-skills warnings
(SKILL.md soft line target; the Plan Review Mode numbered list) are
pre-existing on origin/main, not introduced here.

Co-authored-by: Claude <noreply@anthropic.com>
@claude

claude Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 4m 36s —— View job


Code review complete

  • Gather context (PR diff, CLAUDE.md, changed files)
  • Run skip gate — not applicable, PR is open and has meaningful review surface
  • Review changed files for correctness/maintainability
  • Post findings (inline + summary)

This is a documentation-only change to the planning plugin's interview and plan skills (SKILL.md prompt files, which double as this repo's executable spec, so I reviewed them for instructional consistency, not just prose). Verified the two factual claims the PR body makes against the actual files: the ## Open-question register heading name (interview/templates/checklist.md:14) and the "write at ask-time" rule (interview/context/loop.md:171-173) both check out — the register mechanism described is accurate. The /prototype:explore-directions and /prototype:pressure-test skills referenced from the new mid-interview detour both exist in the repo. The version bump (0.31.0 → 0.32.0) and CHANGELOG entry match the diff.

One finding posted inline:

  • plugins/planning/skills/plan/SKILL.md line 253 — the new "Substantive rounds do not belong in plan mode" text says to route to /planning:interview "run with plan mode off," but doesn't say how, and this conflicts with the section's own established doctrine two lines above (line 249: "the skill does not automatically enter plan mode — the user controls permission modes") — which handles the symmetric entering-plan-mode case by suggesting it to the user, not doing it directly. Without an equivalent "ask the user to exit" step here, a session mid-formulation in plan mode that hits a substantive question has no scripted way to satisfy "plan mode off," and the likely fallback is invoking /planning:interview while still in plan mode — reproducing the exact register-write failure this same paragraph exists to prevent. Full detail and a suggested fix in the inline comment.

Everything else — the new Needs-an-artifact categorization arm in interview/context/loop.md, the gotchas.md addition, and the composition-table row in interview/SKILL.md — is internally consistent and cross-references correctly.

@claude

claude Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 48s —— View job


Security Review

  • Gather context (PR diff, changed files)
  • Run skip gate / assess security-relevant surface
  • Review changed files for security issues
  • Post final findings

Skip gate triggered — no security-relevant surface.

All 6 changed files are documentation-only (Markdown skill instructions + CHANGELOG.md + a plugin.json version bump, 0.31.00.32.0):

  • plugins/planning/.claude-plugin/plugin.json — version bump only
  • plugins/planning/CHANGELOG.md — changelog prose
  • plugins/planning/skills/interview/SKILL.md — new "ungrillable question" mid-interview detour row/paragraph
  • plugins/planning/skills/interview/context/gotchas.md — new gotcha about plan mode blocking the ask-time register write
  • plugins/planning/skills/interview/context/loop.md — new "Needs-an-artifact" categorization arm
  • plugins/planning/skills/plan/SKILL.md — reconciles plan-mode question scope with the interview skill

There is no code, no script, no GitHub Actions workflow, no permission/config change, no dependency/action pin, and no executable logic anywhere in the diff — just prose describing when/how an agent should invoke other skills (/prototype:explore-directions, /prototype:pressure-test, /planning:interview) via the Skill tool. No injection surfaces, no secret handling, no trust-boundary changes, no authorization logic.

No security issues found. Per this lane's high-signal bar, nothing further to report.

…#2998)

The two lane 5 ADOPT rows carried only their `Filed:` pointer. Both are
now delivered by #3045, so each row names the delivering PR and what
actually landed — the prototype detour plus the Needs-an-artifact
categorization arm on one, the plan-mode gotcha plus the `planning:plan`
scoping-confirm reconciliation on the other — and moves its as-of date.

This is the row-home the issue's criteria point at: the originally cited
`docs/upstream/aihero-core-lanes.md` dissolved at the lane 6 harvest and
its lane 5 rows now live here, per the redirect recorded on #2998.

Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Last security-reviewed head: 9b58bfbf754fb6ec353c1f7ebbd5b90ce0dc56f7. On the next push, the relevance gate compares only the commits since this SHA; delete this comment to force a full re-review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 91cbd014d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/planning/skills/interview/SKILL.md Outdated
…e detour (#2998)

Codex review on #3045, verified against the tree: the prototype skills
gate model-initiated entry — stop and confirm scope before writing any
throwaway code, and do not detour out of an active workflow without
checkpointing first. The detour as written invoked them immediately, so
the interview handed off without the confirm and without an explicit
checkpoint of its own live state.

The detour now confirms the spike's scope with the user and checkpoints
the interview before handing off, and names what that checkpoint already
consists of: the question registered `open` at ask-time plus the ledger
and Brief written per lock-in, which is exactly what lets the open branch
survive the detour. The composition-table row and the CHANGELOG entry
carry the same sequence.

The gate is described rather than cited: another skill's `context/` files
are private, so the requirement is stated in the interview's own terms.

Co-authored-by: Claude <noreply@anthropic.com>
@claude

claude Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 43s —— View job


Security Review

  • Gather context (PR diff, changed files)
  • Run skip gate / assess security-relevant surface
  • Review changed files for security issues
  • Post final findings

Skip gate triggered — no security-relevant surface.

Per the lane's skip gate, this run compares only the commits since the last-reviewed head (91cbd014d6843f2af62a7ceecf539b47293a7432) through the current HEAD (5d4212c3c7dee340e4d5a2af063a8a6efca2883c). That delta is three files, all prose:

  • docs/upstream/aihero-course.md — marks the two ADOPT rows "(delivered)" and links this PR
  • plugins/planning/CHANGELOG.md — changelog prose addition describing the prototype-gate fix
  • plugins/planning/skills/interview/SKILL.md — reworks the "ungrillable question" mid-interview
    detour paragraph to route through a scope-confirm + checkpoint step before invoking
    /prototype:explore-directions or /prototype:pressure-test, addressing the earlier Codex
    finding that the detour skipped the prototype skills' own model-initiated entry gate
    (plugins/prototype/context/discipline.md)

No code, no script, no GitHub Actions workflow, no permission/config change, no dependency/action
pin — just Markdown instructing an agent when/how to invoke another skill via the Skill tool. No
injection surfaces, no secret handling, no trust-boundary changes, no authorization logic.

Combined with the earlier pass on 91cbd01 (also a full skip — the whole PR is documentation-only:
skill instructions, a CHANGELOG.md, and a plugin.json version bump), the entire PR head has now
been covered. No security issues found. Per this lane's high-signal bar, nothing further to report.

Comment thread plugins/planning/skills/plan/SKILL.md Outdated
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude has reviewed this PR 3 times. The lane skips further automatic reviews after 5; deleting this comment resets the count.

…#2998)

Code-review finding on #3045, verified against the section: the new
"substantive rounds do not belong in plan mode" text asserted the
interview runs with plan mode off but named no mechanism, and the
paragraph four lines above already settles that the skill toggles no
permission mode — it *suggests* entering and lets the user act.

Asserting the exit case without the symmetric step left the rule
unsatisfiable: a run mid-formulation in plan mode ("stay in it") that
hits a substantive question had no scripted way to reach plan-mode-off,
and the likely fallback — invoking the interview from inside plan mode —
reproduces the exact register-write failure the paragraph exists to
prevent.

The exit is now the user's move, stated the same way as the entry: say
why, ask the user to exit (shift+tab), invoke the interview once they
have, and never invoke it from inside plan mode assuming the register
write will survive. The CHANGELOG entry carries the same clause.

Co-authored-by: Claude <noreply@anthropic.com>
@claude

claude Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 46s —— View job


Security Review

  • Gather context (PR diff, changed files)
  • Run skip gate / assess security-relevant surface
  • Review changed files for security issues
  • Post final findings

Skip gate triggered — no security-relevant surface.

Per the lane's skip gate, this run compares only the commits since the last-reviewed head
(5d4212c3c7dee340e4d5a2af063a8a6efca2883c) through the current HEAD
(19c7cbee29001add9ce5c6cab955ea6af34b916c). That delta is one commit, two files, both prose:

  • plugins/planning/CHANGELOG.md — changelog clause describing the fix below
  • plugins/planning/skills/plan/SKILL.md#L253 — adds the missing mechanism for exiting plan mode before invoking /planning:interview (the correctness gap flagged in the earlier code review): the skill now explicitly asks the user to exit plan mode (shift+tab) rather than asserting "plan mode off" with no scripted way to get there.

No code, no script, no GitHub Actions workflow, no permission/config change, no dependency/action
pin — just Markdown clarifying when/how an agent should ask the user to change permission mode
before invoking another skill via the Skill tool. No injection surfaces, no secret handling, no
trust-boundary changes, no authorization logic.

Combined with the earlier passes on 91cbd01 and 5d4212c (both full skips — the whole PR is
documentation-only: skill instructions, a CHANGELOG.md, and a plugin.json version bump), the
entire PR head has now been covered. No security issues found. Per this lane's high-signal bar,
nothing further to report.

@claude

claude Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 2m 51s —— View job


Code review complete

  • Gather context (PR diff, CLAUDE.md, changed files, prior review rounds)
  • Run skip gate — not applicable; this sync commit specifically targets the one open finding from the prior round
  • Review the delta since the last-reviewed head, and re-check the full diff for consistency
  • Post findings

This lane already reviewed this PR twice (at 91cbd01 and again after 5d4212c). The only open finding was mine, from the 91cbd01 pass: plugins/planning/skills/plan/SKILL.md#L253 said to route to /planning:interview "run with plan mode off" without saying how plan mode gets turned off, given the skill can't toggle permission modes itself — and it broke symmetry with the entering-plan-mode case two lines above, which does script a "suggest to the user" step.

This commit (19c7cbe) fixes it, and the fix is correct. The new sentence at line 253 now reads:

Getting there is the user's move, not yours — symmetric to entering plan mode above: you do not toggle permission modes, so when plan mode is active and a substantive round comes due, say why and ask the user to exit it (shift+tab), then invoke the interview once they have. Do not invoke it from inside plan mode on the assumption the register write will survive — it will not.

This closes the exact gap flagged: it's now symmetric with the entering-plan-mode doctrine at line 249 (suggest, don't act), it names the concrete mechanism (shift+tab), and it makes the failure mode explicit rather than implicit. The matching CHANGELOG entry (plugins/planning/CHANGELOG.md [0.32.0] → Changed) describes the same fix accurately.

No new issues found. I re-checked the full diff against origin/main (all 7 files), not just the incremental commit:

  • The Needs-an-artifact categorization arm added to plugins/planning/skills/interview/context/loop.md:80 isn't hardcoded anywhere else in scripts (confirmed no script greps the category label set), so it can't silently break a gate.
  • The claims in the new plugins/planning/skills/interview/context/gotchas.md:19 gotcha and the SKILL.md:300 "ungrillable question" paragraph — the ## Open-question register heading name and the "write at ask-time" rule — check out against interview/context/loop.md:162 and :171.
  • The earlier Codex/security-lane finding about honoring the prototype skills' own model-initiated entry gate (plugins/prototype/context/discipline.md:14-16) was addressed in 5d4212c and remains intact here — the detour text confirms scope and checkpoints before handing off, and the ledger/Brief-per-lock-in mechanism it names is a legitimate (if non-literal) substitute for the /session-flow:handoff example in that gate, since the interview's own on-disk state already survives a crash/clear per its own design.
  • docs/upstream/aihero-course.md and plugins/planning/.claude-plugin/plugin.json (0.31.0 → 0.32.0) diffs match what the PR body and CHANGELOG describe.

Note: I did not independently re-run the gate scripts (validate-plugins.sh, check-changelog-parity.sh, markdownlint-cli2, etc.) listed in the PR's Verification table — that table is the author's own claimed output, not something I confirmed myself.

Resolves the plugins/planning/CHANGELOG.md conflict: main shipped planning
0.32.0 for #2936 while this branch had claimed the same number for #2998.
The #2998 entries move up to a new 0.33.0 heading above main's preserved
0.32.0 section, and plugin.json bumps to 0.33.0 to match.

plan/SKILL.md auto-merged cleanly — both #2936's test-boundary naming and
this branch's plan-mode routing are present.
@claude

claude Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 29s —— View job


Code review complete

  • Gather context (PR diff, CLAUDE.md, changed files, prior review rounds)
  • Run skip gate — skip gate triggered (see below)
  • Review the delta since the last-reviewed head
  • Post findings

Skip gate triggered — condition 4 (this head already has a successful review from this lane that still applies).

This lane has already reviewed this PR three times, at 91cbd01, 5d4212c, and 19c7cbe — the last of which fixed the only open finding (the plan/SKILL.md "run with plan mode off" mechanism gap) and was confirmed correct in the follow-up pass.

The only commit since 19c7cbe is 9b58bfb, a merge of origin/main into this branch. I diffed origin/main...HEAD (the actual net PR diff, matching the task's reported 7 files / +46/-5) and separately diffed 19c7cbe..HEAD on the PR's own files to isolate what the merge changed:

  • plugins/planning/skills/interview/SKILL.md, context/gotchas.md, context/loop.md, docs/upstream/aihero-course.mdbyte-identical to the already-reviewed 19c7cbe versions.
  • plugins/planning/skills/plan/SKILL.md — the PR's own content (the line-253 "Getting there is the user's move" fix) is unchanged; the only diff since 19c7cbe is an unrelated "Name the test boundaries" paragraph at line 93 that arrived from origin/main (PR Lane C: TDD wiring + zero-assembly default chain #2936), already merged and reviewed independently on main.
  • plugins/planning/.claude-plugin/plugin.json / CHANGELOG.md — version bump shifted 0.32.00.33.0. This is a correct, mechanical rebase: origin/main had already claimed 0.32.0 via the unrelated Lane C: TDD wiring + zero-assembly default chain #2936 merge (confirmed via git show origin/main:plugins/planning/CHANGELOG.md), so this branch's own [0.32.0] heading was renumbered to [0.33.0] with its original content preserved verbatim, and the prior [0.32.0] (from main) sits below it intact.

No new PR-owned content was introduced by the merge, so there is nothing new to review. Per this lane's skip gate, posting nothing further.

@claude

claude Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 39s —— View job


Security Review

  • Gather context (PR diff, changed files, prior review rounds)
  • Run skip gate / assess security-relevant surface
  • Review changed files for security issues
  • Post final findings

Skip gate triggered — no security-relevant surface, and this head's diff already fully reviewed.

HEAD 9b58bfb is a merge of origin/main into this branch (bringing in unrelated main-line progress, e.g. plugins/planning/skills/plan/context/plan-template.md and the audit-skill-visibility skill) plus one unrelated main commit (d1c0935). Neither touches this PR's own change set.

Comparing this PR's actual contribution against the current origin/main (git diff origin/main...HEAD, merge-base d1c0935) shows the same 7 files as before, and the added content is byte-for-byte the same prose already covered by the three prior full passes on this PR (91cbd01, 5d4212c, 19c7cbe):

The only mechanical difference from the previously reviewed state is that the merge rebased this PR's version bump onto a newer origin/main baseline (main independently landed its own 0.32.0 in the meantime for an unrelated change, so this PR's bump now reads 0.32.00.33.0 in the diff instead of 0.31.00.32.0) — a renumbering artifact of the merge, not new content, and not security-relevant.

As established across the earlier passes: this is a documentation-only change (Markdown skill instructions, a CHANGELOG.md, a plugin.json version bump, and one upstream-tracking doc). No code, no script, no GitHub Actions workflow, no permission/config change, no dependency/action pin, no injection surface, no secret handling, no trust-boundary or authorization logic anywhere in the diff.

No security issues found. Per this lane's high-signal bar, nothing further to report.

@kyle-sexton
kyle-sexton merged commit d9f15d4 into main Aug 20, 2026
51 checks passed
@kyle-sexton
kyle-sexton deleted the claude/pocock-steering-course-00zkvd branch August 20, 2026 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

planning: interview prototype detour + plan-mode-off guidance (aihero lane 5 audit findings)

2 participants