Skip to content

feat(planning): interview asks intent and quality requirements by quality bar - #8407

Merged
kyle-sexton merged 11 commits into
mainfrom
feat/8333-interview-intent-and-quality-requirements
Oct 19, 2026
Merged

kyle-sexton merged 11 commits into
mainfrom
feat/8333-interview-intent-and-quality-requirements

Conversation

@kyle-sexton

Copy link
Copy Markdown
Member

Closes #8333

Summary

In an engineering session /planning:interview now reads the consumer's standards before any bundled default, asks the intent question (is a second variant known, a category, or speculative; what the existing structure does; what upstream context says), and prompts quality requirements (NFRs) from a new prompt list, each scaled by the quality bar. The Brief records Quality bar: and Intent: lines under Captured assumptions and each quality requirement as a Quality requirement: line under Constraints.

Fix

  • plugins/planning/skills/interview/context/loop.md gains an "Engineering defaults" section; context/nfr-prompts.md is the new prompt list; SKILL.md points to the section after the Step 1 survey.
  • Q47 interpretation, for review:
    • A confirmed Brief is one whose Restatement: line reads confirmed by the user.
    • Text a bot or a teammate wrote informs the recommendation and never settles intent.
    • A refactor that extracts a new abstraction asks the full intent question.
    • Unattended, the bug-fix or refactor collapse applies only when the user's own words name a bug fix or a behavior-preserving refactor.
  • lock rule: a quality prompt at the bar's depth that the request does not cover becomes a captured assumption with a revisit trigger (or is stated as not applying), never a gap that halts the run.
  • Consumer standard wins: when a consumer standard contradicts a bundled default, the Brief template records - Overrides bundled default <principle> per <standards file> under Captured assumptions.
  • Blocker last-line shape: unattended, an unstated intent is recorded blocked (arbiter USER-RESERVED), the Brief is still written, and the run's last line is Blocked: Q<N> <question> (arbiter: USER-RESERVED).
  • Upstream context is recorded as a short summary; private titles and URLs are left out in public or unknown-visibility repositories.
  • Five new eval cases under the planning plugin's evals; changelog fragment .changes/planning/feat-8333-interview-intent-and-quality-requirements-4c7d2a91.md (minor).
  • Not done: no pointer was added inside SKILL.md Step 1. That section is digest-pinned by plugins/planning/tests/interview-defenses.test.sh, and SKILL.md:45 already points to "Engineering defaults".

Verification

  • Paid evals, branch plugin vs main plugin, 3 runs per arm. The noise report says n is too small to call any of these.

    Case Round Branch Main
    nfr-quality-requirement (unattended) r4 1.00 0.00
    consumer-standard-override (unattended) r4 1.00 0.50
    intent-unattended-blocked r2 1.00 0.58
    intent-asked-recorded r2 1.00 0.00
    intent-prototype-one-line r1 0.78 0.67
  • Two cases run unattended because an attended one-prompt lock correctly stops on open domain questions, so it cannot test what the Brief contains.

  • bash scripts/check-affected.sh --tests --base origin/main: 8 passed, 1 skipped over 47 changed files.

  • typos --format brief on the changed paths: clean.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 9 commits October 18, 2026 23:24
One directive line per foundational principle (SRP, OCP, LSP, ISP, DIP,
composition over inheritance, DRY of knowledge, coupling and cohesion,
information hiding, deep modules and change amplification, YAGNI's
cheap-now test, reuse first), each cited to its defining source with an
as-of date and a recheck trigger. It opens with the scope line and the
consumer-wins override line, and points at /review:quality-gate
code-design for review-time criteria.

Refs: #8332

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lity bar

In an engineering session the interview now reads the consumer's standards
before any bundled default, asks the intent question (second variant known,
category or speculative; existing structure; upstream), and prompts quality
requirements (NFRs) from a new context/nfr-prompts.md, each scaled by the
quality bar. The Brief records Quality bar: and Intent: lines under Captured
assumptions, Quality requirement: lines under Constraints, and an
"Overrides bundled default ... per ..." line when a consumer standard wins.
Unattended, only the user's own words settle the intent question; an
unstated answer is blocked with arbiter USER-RESERVED and the Brief is still
written. Upstream context is recorded as a short summary that leaves private
titles and URLs out of public repositories.

The text lives in context/loop.md; SKILL.md gains one pointer sentence and
keeps its 499 lines. Adds five eval cases and a changelog fragment.

Refs #8333

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add Stumble lines to the unattended intent rules and the quality-requirement
rules, point the quality-bar stumble at /review:quality-gate code-design, and
drop the /planning:design consumer claims. Cross-reference the deferred intent
row to the unattended stop and unconfirmed report, repeat the prototype
ambient-standards exception, and show the bug-fix form in the Brief template.

Tighten the reliability grader to require the degraded-provider behavior and
the second-variant grader to require the variant term inside the question,
with fail samples for both false positives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… blocker grader

Under lock, a quality prompt the request does not cover becomes a captured
assumption with a revisit trigger or is stated as not applying, never a gap
that halts. The Brief template carries the bundled-default override line.
The unattended intent blocker is the run's last line in a fixed shape.

The names-blocker grader targets last_message explicitly, narrows the
negation guard to words directly before "block", lets the USER-RESERVED tag
sit on the next line, and runs offline under Python and JavaScript. Two
grader regexes drop fragments typos flagged, and the performance grader
accepts "300 milliseconds".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…widen two graders

The NFR and consumer-standard cases now say which Loqate result counts as
undeliverable, where the Pay button lives, and (for the standard case)
that checkout fails open, so lock has no domain gap to halt on. No graded
expectation changes.

emits-override-line allows a principle up to 160 characters, and
reads-standard is a trace regex that accepts the standards file read by
Read or by cat, sed, head or tail in Bash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntent-and-quality-requirements

# Conflicts:
#	plugins/planning/reference/design-principles.md
Both cases now declare the run unattended in the wording the
unattended-blocked case uses, so a domain unknown becomes a captured
assumption or a blocked row and the Brief is still written. The graders
then measure the quality-requirement and override lines rather than lock
halting on the next open question.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review October 19, 2026 07:23
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T07:25:15.500936Z a3baffb Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 15c8314fa139c7eba0826259498791a3b317e7da: 0 finding(s), 0 as inline comments.

Security review (incremental, fa45f596..15c8314f): no security issues found.

The only change is one line in plugins/planning/skills/interview/context/loop.md:77. It splits the lock gap rule by mode. Attended runs still "STOP and surface it". Unattended runs follow the Unattended bullets.

Because the edit narrows a standing instruction, I read it as an instruction-surface change. I checked whether the Unattended path still enforces the same control, using the file's own text. I did not read docs/conventions/instruction-exception-register/README.md, so I did not check the change against its protected-class list.

  • Unattended handling: the Unattended bullets (loop.md:108-119) and the ladder (loop.md:386-396) cover a missing or upstream-only part 1 answer. The row is recorded blocked with arbiter: USER-RESERVED. The Brief goes to ### Deferred questions and the run ends on a Blocked: Q<N> line. The run does not assume the answer or hold the lane. The contract is reported unconfirmed.
  • No new assumption path: the Unattended bullets say only the user's own words, or a Brief or PRD the user confirmed, can settle part 1. Text from a bot or a teammate cannot, and conflicting sources count as unstated. That is stricter than "upstream only", so injected upstream content cannot settle a user-reserved decision.
  • Auto-guard preserved: the ladder keeps the rule against silently folding a user decision into the Brief as an assumption (loop.md:398).

The reworded rule keeps its control, in both attended and unattended runs. No finding is warranted.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3baffbb93

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 15c8314fa139c7eba0826259498791a3b317e7da: 0 finding(s), 0 as inline comments.

Incremental review of plugins/planning/skills/interview/context/loop.md since fa45f59: no findings.

The change is one line, in the lock paragraph of "Engineering defaults". The gap sentence now reads "attended, STOP and surface it; unattended, the Unattended bullets below apply."

  • It fixes a conflict. The old text said STOP on a gap, which contradicted the Unattended bullets. Those bullets record the intent row blocked and write the Brief anyway. Before this change, an unattended lock run hitting a missing or contradictory part 1 answer had two incompatible instructions.
  • The pointer resolves. The **Unattended** bullets sit at lines 108–123, after the changed line. They cover the missing and contradictory cases: "Part 1 unstated" → blocked row, and "sources that conflict count as unstated".
  • Both modes are covered. Attended runs keep the STOP behavior and unattended runs follow the bullets, so neither path is left undefined.
  • Not a finding. The new line runs about 150 characters while the paragraph wraps near 100. Lines 56, 115 and 117 of the same file already run longer, and no linter catches it, so it falls below the bar.
  • Closing issue feat(planning): interview asks intent and quality requirements, reads consumer standards #8333. The incremental change doesn't touch any of its acceptance criteria. I checked only this diff and the surrounding file. I did not re-review files reviewed at the earlier SHA, and I did not run the tests or evals. The PR body's verification numbers are author-claimed and unverified by me.

…e-asking

Under lock, a part 1 intent answer the user's own request states is recorded
directly; one stated only upstream, missing, or contradictory stays a gap that
halts the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unstated part 1 under unattended lock now follows the Unattended bullets
(blocked, USER-RESERVED, Brief written unconfirmed); attended lock still stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton added this pull request to the merge queue Oct 19, 2026
Merged via the queue into main with commit 3da9596 Oct 19, 2026
25 checks passed
@kyle-sexton
kyle-sexton deleted the feat/8333-interview-intent-and-quality-requirements branch October 19, 2026 08:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(planning): interview asks intent and quality requirements, reads consumer standards

1 participant