Skip to content

docs(work-items): require brief-before-ask restatement in interactive direction gates - #1367

Merged
kyle-sexton merged 8 commits into
mainfrom
docs/1202-brief-before-ask-direction-gate
Jul 25, 2026
Merged

docs(work-items): require brief-before-ask restatement in interactive direction gates#1367
kyle-sexton merged 8 commits into
mainfrom
docs/1202-brief-before-ask-direction-gate

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Closes #1202

Summary

triage's interactive direction gate and attend-queue's row-working loop could pose an
operator-facing decision question with option labels only, giving no restated item context —
forcing the operator to halt the pass and ask which item is even under discussion. Both skills now
require restating, before any operator decision question, (1) which item, (2) the decision being
asked, and (3) the consequence of each option:

  • triage's interactive direction gate (both the initial recommendation and each step-4 interview
    question).
  • attend-queue's row-working loop ([intake], [escalated], [ratify] rows).

Bumped work-items 0.24.2 → 0.25.0 with a matching CHANGELOG.md entry, and added one new eval
per skill covering the requirement.

Test plan

  • jq empty on both modified evals.json files — valid JSON.
  • Structural validation of both evals.json files against the skill-quality evals schema shape
    (skill_name, non-empty evals, each case has id/prompt/name/expected_output/files/expectations).
  • skill-quality:check (check-skill.sh) run against both triage and attend-queue —
    both PASS (0 errors; triage carries one pre-existing advisory warning, no Gotchas surface).
  • This is a skill-text/docs-only change (no scripts or executable code touched) — no build/lint
    pipeline applies beyond the above.

Related

Sibling observation from the same triage pass that surfaced this item: #1201.

🤖 Generated with Claude Code

@kyle-sexton @claude
docs(work-items): require brief-before-ask restatement in interactive…
4fbf51e 
… direction gates

triage's interactive direction gate and attend-queue's row-working loop could
pose an operator decision question with option labels only, forcing the
operator to halt the pass and ask which item is even under discussion. Both
now require restating (1) which item, (2) the decision being asked, and (3)
the consequence of each option before any operator-facing decision question
- triage's recommendation gate and each step-4 interview question, and
attend-queue's [intake]/[escalated]/[ratify] rows.

Closes #1202

Co-Authored-By: Claude Sonnet 5 (Claude Code) <noreply@anthropic.com>
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 24s —— View job

PR Review

  • Gather context and understand the request
  • Read changed files and examine the diff
  • Review skill text changes for correctness and alignment
  • Review eval additions for quality and coverage
  • Check version bump and changelog
  • Post findings

Overall: This is a well-scoped, clean change. The requirement is correctly positioned in both skills, the cross-references are consistent, and both evals exercise the non-obvious failure mode (context not carrying over between rows) rather than just the happy path.

Findings

triage/SKILL.md — direction gate (line 103) and interview step (line 119)

The direction-gate expansion correctly sandwiches the brief requirement between the "how triage was invoked" framing and the pre-existing wait-for-direction sentence. The added note — "The restatement is not optional compression fodder" — is the right kind of defensive prose: models operating under terse-output instructions tend to compress exactly this kind of context.

The step-4 interview addition correctly scopes the same restatement to follow-up questions, which would otherwise be an unaddressed gap. One minor phrasing observation: the step-4 sentence uses "which item it concerns, the decision being asked, and the consequence of each answer" (emphasis mine), shifting "option" → "answer" to reflect the interview-question context. This is appropriate and accurate, not an inconsistency.

attend-queue/SKILL.md — "Brief before asking" section (lines 68–74) and row bullets (lines 77, 80, 85–87)

The new preamble block cleanly owns the rule once, and each row bullet references it by name rather than restating it — good. One subtle observation worth calling out for reviewers:

The [intake] bullet reads "brief before asking, recommend, wait for direction, then mutate" — it weaves the brief into triage's own flow description rather than just saying "restate the brief above". This is intentional (the complete flow is owned by triage's direction gate, not by attend-queue), but it means the brief is referenced two ways in this section: as "the brief above" for [escalated] and [ratify], and as an inline flow item for [intake]. Both are correct; there's no behavioral ambiguity.

The [ratify] bullet adds "the consequence of ratifying versus declining" — this is a precise, row-specific expansion of the generic "consequence of each option" language, and it's the right level of specificity for a row type with a binary outcome.

triage/evals/evals.json — eval id 7

expected_output covers both the initial recommendation brief and the step-4 interview question in a single scenario. The fifth expectation ("Still waits for the operator's explicit direction before mutating anything") guards against the brief being mistaken as a substitute for the wait — an important correctness boundary.

The prompt correctly seeds the scenario with "you have just recommended a target state" to put the model at the exact transition point where both the direction gate and the interview question are exercisable in one session.

attend-queue/evals/evals.json — eval id 3

The two-row back-to-back scenario (escalated → ratify, different items) is the exact failure mode this PR is defending against, and it's well chosen. The second expectation — "Repeats the same full restatement for the very next row rather than assuming context carries over from the prior row in the same pass" — targets the subtle case where a model might correctly brief the first row and then skip the second on the grounds that "the operator was just told this." The fourth expectation ("the brief precedes the ask, it does not replace it") mirrors the analogous guard in the triage eval.

Version and changelog

0.24.2 → 0.25.0 is correct — this is a feature addition (new behavioral requirement), not a patch. The changelog entry accurately summarizes the scope, names the issue (#1202), and explains the pre-existing operator UX problem being solved.

No blockers. The prose is clear, internally consistent, and the evals cover the meaningful edge cases. Approving.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 50s —— View job

Security Review

  • Gather context and understand the request
  • Fetch PR diff
  • Read changed files
  • Analyze for security issues
  • Post findings

No security issues found.

This PR is a skill-text and evaluation-data-only change. All six modified files were reviewed:

File Change Security surface
plugin.json Version bump 0.24.2 → 0.25.0 None
CHANGELOG.md Prose entry None
attend-queue/SKILL.md AI instruction text (brief-before-ask requirement) None
attend-queue/evals/evals.json New eval case; "files": [] None
triage/SKILL.md AI instruction text (same requirement) None
triage/evals/evals.json New eval case; "files": [] None

Checks performed:

  • Prompt/template injection — The new SKILL.md prose and eval prompts contain only operator-guidance text. No ${...} interpolations are introduced, and no user-supplied data is embedded in positions that reach a shell, SQL engine, or external renderer.
  • Instruction poisoning — The skill text is maintained by the repo owners and reviewed normally through the PR process; the new paragraphs add routine behavioral constraints with no unintended authority grants or capability expansions.
  • Data egress — The [escalated] and [ratify] restatement clauses instruct the model to surface item context to the attending operator in the same session. No new external destinations are introduced.
  • Secrets / credentials — None present or referenced.
  • GitHub Actions — No workflow files touched; no pull_request_target, workflow_run, script-injection, permission-widening, or pin-loosening changes.
  • Supply chain — No dependency, action, or package reference changes.
  • evals.json files field — Both new eval entries use "files": []; no file paths that could be abused for path traversal or sensitive-file reads during eval execution.

@kyle-sexton kyle-sexton mentioned this pull request Jul 25, 2026
Closed
1 task
@kyle-sexton
Merge remote-tracking branch 'origin/main' into docs/1202-brief-befor…
7ebd863 
…e-ask-direction-gate

# Conflicts:
#	plugins/work-items/.claude-plugin/plugin.json
#	plugins/work-items/CHANGELOG.md
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 58s —— View job

PR Review

  • Gather context and understand the request
  • Read changed files and examine the diff
  • Review skill text changes for correctness and alignment
  • Review eval additions for quality and coverage
  • Check version bump and changelog
  • Post findings

Overall: This is a well-scoped, clean change. The synchronize event reflects a merge with origin/main; the PR's content is unchanged. The brief-before-ask requirement is correctly positioned in both skills, cross-references are consistent, and the evals exercise the non-obvious failure modes rather than just the happy path. No blockers.

triage/SKILL.md — direction gate (line 103) and interview step (line 119)

The direction gate expansion sandwiches the brief requirement cleanly between the invocation-mode framing and the pre-existing wait-for-direction sentence. The added line — "The restatement is not optional compression fodder: a terse output style must never drop it, and it applies on every decision question, not only the first one of a pass" — is precisely the kind of defensive prose that matters for models inclined to compress context under terse-output instructions.

The cross-reference to attend-queue at the end of that sentence ("the operator working several rows in sequence (e.g. via /work-items:attend-queue) cannot be assumed to still be holding a prior item's context") is a good addition — it makes explicit the exact multi-row use case this requirement defends against, which is otherwise only implied by the "not only the first one" language.

The step-4 interview addition correctly closes the gap for follow-up questions, which would otherwise be an unaddressed site. The shift from "consequence of each option" (direction gate) to "consequence of each answer" (interview context) is appropriate and accurate — not an inconsistency.

attend-queue/SKILL.md — "Brief before asking" section (lines 68–74) and row bullets

The new preamble block correctly owns the rule once and covers all three row kinds explicitly ([intake], [escalated], [ratify]). Each row bullet then references it by name rather than restating it inline — good ownership structure.

One thing worth calling out for reviewers: the [intake] bullet uses "brief before asking, recommend, wait for direction, then mutate" rather than the "restate the brief above" phrasing used by [escalated] and [ratify]. This is correct — for intake rows, the brief-before-ask is delegated into triage's own interactive direction gate (which already owns that flow), so attend-queue describes the complete flow rather than owning a separate restatement. No ambiguity in the behavioral outcome.

The [ratify] precision — "the consequence of ratifying versus declining" — is the right level of specificity for a row type with a binary outcome.

triage/evals/evals.json — eval id 7

The new eval seeds the scenario at the exact transition point ("you have just recommended a target state") to exercise both the direction gate and the step-4 interview question in one session. All five expectations are concrete and testable:

  • The fourth expectation guards against compression under a terse output style.
  • The fifth — "Still waits for the operator's explicit direction before mutating anything" — guards against treating the restatement as a substitute for the wait, which is the critical correctness boundary.

attend-queue/evals/evals.json — eval id 3

The two-row back-to-back scenario (escalated → ratify, different items) is the exact failure mode this PR defends against. The second expectation — "Repeats the same full restatement for the very next row rather than assuming context carries over from the prior row in the same pass" — targets the subtle case where a model correctly briefs the first row and then skips the second on the grounds that the operator was just told this. The fourth expectation mirrors the analogous guard from the triage eval.

Version bump and changelog

0.24.2 → 0.25.0 is correct — this is a feature addition (new behavioral requirement), not a patch. The changelog entry accurately summarizes the scope, names the issue (#1202), and explains the pre-existing operator UX problem being solved.

No issues found. Approving.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude finished @kyle-sexton's task in 1m 17s —— View job

Security Review

  • Gather context and understand the request
  • Fetch PR diff
  • Read changed files
  • Analyze for security issues
  • Post findings

No security issues found.

This PR is a skill-text and evaluation-data-only change. All six modified files were reviewed:

File Change Security surface
plugins/work-items/.claude-plugin/plugin.json Version bump 0.24.2 → 0.25.0 None
plugins/work-items/CHANGELOG.md Prose entry None
plugins/work-items/skills/attend-queue/SKILL.md AI instruction text (brief-before-ask preamble + row references) None
plugins/work-items/skills/attend-queue/evals/evals.json New eval case (id 3); "files": [] None
plugins/work-items/skills/triage/SKILL.md AI instruction text (direction gate + step-4 interview expansion) None
plugins/work-items/skills/triage/evals/evals.json New eval case (id 7); "files": [] None

Checks performed:

  • Prompt/template injection — The new SKILL.md prose and eval prompts contain only operator-guidance text. No ${...} interpolations are introduced, and no user-supplied data is embedded in positions that reach a shell, SQL engine, or external renderer.
  • Instruction poisoning — The skill text is maintained by repo owners and reviewed through the normal PR process. The new paragraphs add behavioral constraints (restate context before asking) with no unintended authority grants or capability expansions.
  • Data egress — The new brief-before-ask restatement clauses instruct the model to surface item context to the attending operator in the current session. No new external destinations are introduced.
  • Secrets / credentials — None present or referenced.
  • GitHub Actions — No workflow files touched; no pull_request_target, workflow_run, script-injection, permission-widening, or pin-loosening changes.
  • Supply chain — No dependency, action, or package reference changes.
  • evals.json files field — Both new eval entries use "files": []; no file paths that could be abused for path traversal or sensitive-file reads during eval execution.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 25, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ebd863f8c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/work-items/skills/triage/SKILL.md Outdated
Comment thread plugins/work-items/skills/triage/evals/evals.json Outdated
@kyle-sexton
fix(work-items): scope brief-before-ask consequences to presented opt…
64a146f 
…ions

Requiring "the consequence of each answer" was unsatisfiable for an
open-ended scoping question: the answer set is unbounded, so a compliant
agent had to either invent a closed option list or drop part of the
restatement. Consequences are now scoped to the options actually
presented, with an explicit rule that an open-ended question states what
the answer will determine instead - and never gets narrowed to satisfy
the restatement.

Eval 7's scenario also began after the recommendation had been presented
while its expectations required a restatement before it, so a model could
satisfy it by replaying a completed step. The scenario now starts before
the recommendation, which is the behavior the fixture exists to cover.
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@github-actions

github-actions Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown

Warning

Automated review did not complete — this is an infrastructure failure, not a review verdict.

Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."

Re-running the job, or pushing a new commit, will retry the review.

@github-actions

github-actions Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown

Warning

Automated security review did not complete — this is an infrastructure failure, not a review verdict.

Treat any Claude comment on this PR (including a placeholder like "I'll analyze this and get back to you") as incomplete, not "no findings."

Re-running the job, or pushing a new commit, will retry the review.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 25, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 64a146f1eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/work-items/skills/attend-queue/evals/evals.json Outdated
Comment thread plugins/work-items/skills/triage/evals/evals.json Outdated
@kyle-sexton
test(work-items): ground the brief-before-ask fixtures in reachable s…
9794655 
…cenarios

The triage fixture asked one turn to demonstrate both the
pre-recommendation brief and the step-4 interview brief, but a correct
implementation stops at the direction gate and never reaches step 4, so
expectations 3-4 were unreachable. Split into eval 7 (pre-recommendation
brief, ends at the gate) and eval 8 (post-direction interview question,
which is where the open-ended carve-out is actually exercised).

The attend-queue fixture named no item numbers, titles, options, or
intended dispatch, so a model had to invent the very context the
expectations require it to restate - a hallucinated brief scored the
same as a grounded one. Both rows now carry concrete markers, and a new
expectation forbids inventing detail the markers do not supply.
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 25, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9794655a6f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/work-items/skills/attend-queue/evals/evals.json Outdated
Comment thread plugins/work-items/skills/triage/evals/evals.json Outdated
@kyle-sexton
test(work-items): put both brief-before-ask fixtures at reachable wor…
a4925b1 
…kflow positions

Triage eval 7 claimed verification was already finished while the
recommendation had not yet been presented, but the direction gate is
step 2 and verification is step 3 - no compliant run reaches that state.
The prompt now places the run at the step-2 gate, before verification.

Eval 8 likewise needed the enhancement framing that makes step 4
reachable without verification, so it now says so.

Attend-queue eval 3 supplied both markers but never the operator's
answer to the first row, so a compliant run had to stop at the wait and
could never reach the second row. The operator's answer to #418 is now
part of the scenario, with an expectation that the wait is honored
rather than self-answered.
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 25, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4925b1488

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/work-items/skills/attend-queue/evals/evals.json
@kyle-sexton
test(work-items): supply the ratify response the attend-queue fixture…
23d4e0c 
… waits on

The scenario named #503's intended dispatch and both outcomes but never
the operator's ratify-or-decline answer, so recording the ratification
required presuming it from the framing - the same missing-turn defect
the #418 row had. The operator's "ratified" response is now part of the
scenario, and the expectation covers both waits.
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Jul 25, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 23d4e0cbe2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/work-items/skills/triage/SKILL.md Outdated
@kyle-sexton
fix(work-items): scope the direction gate's consequences to presented…
72aed95 
… options

The step-4 interview already said "each option you present"; the
direction gate above it still said "each option", which reads as every
target state the state machine admits rather than the recommendation and
the alternatives actually put to the operator.
@kyle-sexton
Merge remote-tracking branch 'origin/main' into docs/1202-brief-befor…
f04d08d 
…e-ask-direction-gate

# Conflicts:
#	plugins/work-items/.claude-plugin/plugin.json
#	plugins/work-items/CHANGELOG.md
@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@claude

claude Bot commented Jul 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Claude encountered an error after 2s —— View job

I'll analyze this and get back to you.

@kyle-sexton
kyle-sexton merged commit 6bd538f into main Jul 25, 2026
26 checks passed
@kyle-sexton
kyle-sexton deleted the docs/1202-brief-before-ask-direction-gate branch July 25, 2026 23:24
@kyle-sexton kyle-sexton mentioned this pull request Jul 26, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

work-items(attend-queue/triage): interactive direction gate lacks a brief-before-ask requirement — operator asked to decide without item context

1 participant

@kyle-sexton