feat(planning): interview recommends downstream model, effort, and advisor - #750
Conversation
…visor The interview already reads task complexity and ambiguity to drive its rounds. At the stop/handoff boundary it now turns that read into a recommendation for the downstream execution session: a model tier (capability), an effort level (thoroughness), and the advisor pairing when the main model is a faster tier — picked per the official capability-vs-thoroughness distinction. Current model names and accepted pairings are read live from the official docs each run and never pinned (durable distinction stable, names drift), mirroring draft-goal-condition's live-doc discipline. A doc-fetch failure degrades to the durable distinction with a visible note rather than halting or guessing a name. Advisory only, fires for engineering and general sessions, and carries an inverse mid-task direction. Closes #231 Co-Authored-By: Claude Opus <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 19s —— View job Security Review
No security issues found. All six changed files are markdown skill instructions, a JSON eval, and a version bump. There is no executable code in this diff. Files reviewed:
External URL fetching (prompt injection surface — assessed as acceptable): The trust boundary here is Anthropic's own infrastructure, which is the same level already accepted by the sibling No other surfaces: no shell templating, no path construction, no secrets handling, no workflow files, no dependency changes, no access-control modifications. |
|
Claude finished @kyle-sexton's task in 1m 55s —— View job PR Review — feat(planning): interview recommends downstream model, effort, and advisor
SummaryThe implementation is sound and follows established patterns in this plugin. The design correctly separates two orthogonal knobs (capability vs. thoroughness), mirrors the Three findings below, ordered by severity. Findings1. URL format inconsistency with CLAUDE.md canonical table (low)
The CLAUDE.md canonical URL table uses the suffix-free form throughout (e.g. 2. Naming Sonnet+Opus inline creates a training-time anchor despite the warning (low)
As of the contract this skill targets, that pairing is **Sonnet main + Opus
advisor** — but the current model names and which pairings are accepted are exactly
the values that drift, so source them live (below), never from this sentence.The "never from this sentence" guard is clever but fragile: a model reading this at recommendation-time will have both the concrete names and the instruction to fetch live in context. Under normal operation (successful fetch) the live names win. Under a fetch failure the graceful-degrade path explicitly falls back to the durable distinction — not to the inline names — which is correct. The risk is a partially-degraded path where the fetch returns a network error but the model still emits "Sonnet + Opus" from this sentence rather than the described durable distinction. Since the inline names are already in the PR description's "sources verified" note and the PR author accepted the tradeoff, this is a watch item rather than a blocker — but tightening the degrade wording in 3. Graceful-degrade path is asserted but not isolated in evals (observation)
Positive notes
|
|
Claude finished @kyle-sexton's task in 1m 22s —— View job Security Review — feat(planning): interview recommends downstream model, effort, and advisor
No security issues found. All six changed files are markdown skill instructions, a JSON eval, and a version bump. There is no executable code in this diff. Files reviewed
Prompt-injection surface — doc fetching (assessed: acceptable)The new All four are Anthropic-controlled domains. The fetched content informs advisory text shown to the user — it does not flow into shell commands, file writes, file paths, or tool arguments. An attacker exploiting this surface would need to have already compromised Anthropic's documentation infrastructure, which is outside the threat model for a plugin PR review. This is identical in character to the already-accepted Other surfaces — all clear
|
|
Claude finished @kyle-sexton's task in 2m 43s —— View job PR Review — feat(planning): interview recommends downstream model, effort, and advisor
SummaryThe implementation is correct and well-structured. The two-knob framing (capability vs. thoroughness), live-doc discipline, graceful degrade, and advisory framing are all handled properly. The hub/spoke split between Findings1. URL suffix inconsistency — fixed locally, push blocked (low)
The CLAUDE.md canonical URL table uses the suffix-free form throughout (e.g. - `https://code.claude.com/docs/en/model-config.md` — model aliases and the effort setting
+ `https://code.claude.com/docs/en/model-config` — model aliases and the effort setting
`https://claude.com/blog/claude-model-and-effort-level-in-claude-code` — which model and effort fit which work
- `https://code.claude.com/docs/en/advisor.md` — advisor enablement and accepted main+advisor pairings
+ `https://code.claude.com/docs/en/advisor` — advisor enablement and accepted main+advisor pairings2. Inline model name creates a training-time anchor despite the guard (watch item)
As of the contract this skill targets, that pairing is **Sonnet main + Opus
advisor** — but the current model names and which pairings are accepted are exactly
the values that drift, so source them live (below), never from this sentence.The 3. Graceful-degrade eval assertion is not isolatable (observation)
Positive notes
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2203a9e549
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Two review findings on the session-config recommendation spoke: - Remove the inline "Sonnet main + Opus advisor" pairing. Eval id 9 asserts names are "not pinned in the skill" and the degrade path must "not guess a model name"; the pinned illustration contradicted both and left a stale fallback anchor under a partial-degrade fetch failure. The durable faster-main + stronger-advisor *shape* stays; the drifting names are sourced live only. - Drop the `.md` suffix from the two code.claude.com/docs URLs to match the suffix-free canonical URL convention in CLAUDE.md (L15-29) and the sibling draft-goal-condition skill. Co-Authored-By: Claude Opus <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 35s —— View job Security Review — feat(planning): interview recommends downstream model, effort, and advisor
No security issues found. All six changed files are markdown skill instructions, a JSON eval, and a version bump. There is no executable code introduced by this diff. Files reviewed
Prompt-injection surface — doc fetching (assessed: acceptable)
All four are Anthropic-controlled domains. Fetched content informs advisory text shown to the user — it does not flow into shell commands, file writes, file paths, or tool arguments. An attacker exploiting this surface would need to have already compromised Anthropic's documentation infrastructure, which is outside the threat model for a plugin PR review. Note: the latest commit ( This surface is identical in character to the already-accepted Other surfaces — all clear
|
|
Claude finished @kyle-sexton's task in 2m 10s —— View job PR Review — feat(planning): interview recommends downstream model, effort, and advisor
SummaryThe implementation is correct and the fixup commit ( Fixup commit verificationThe prior reviews flagged two issues; both are resolved at HEAD: 1. Pinned advisor names removed —
The "Sonnet main + Opus advisor" anchor is gone. Only the durable shape remains. Eval id 9's 2. URL suffix fixed — both Observations1. Mid-task inverse direction is advisory text, not an automated signal (watch item)
The Codex review flagged that no mechanism automatically loads this context during an execution session — the inverse-direction guidance is exercised only if the user explicitly invokes No fix required — it's useful guidance and doesn't create broken behavior. But if the framing gets a future tightening pass, adding "if this skill is invoked mid-execution" would remove the ambiguity about when the guidance applies. 2. Graceful-degrade eval branch is asserted but not isolatable (observation, pre-existing)
"On a doc-fetch failure the skill degrades to the durable distinction with a visible note and does not halt the interview or guess a model name"This expectation can only be exercised if Positive notes
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be3590c95b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressing the claude[bot] PR review findings (commit
CI is green; PR is BLOCKED pending required human review approval. |
…el-effort-recommend # Conflicts: # plugins/planning/.claude-plugin/plugin.json # plugins/planning/CHANGELOG.md
|
Claude finished @kyle-sexton's task in 1m 33s —— View job Security Review — feat(planning): interview recommends downstream model, effort, and advisor
No security issues found. All six changed files are markdown skill instructions, a JSON eval, and a version bump. No executable code is introduced by this diff. Files reviewed
Prompt-injection surface — doc fetching (assessed: acceptable)
All four are Anthropic-controlled domains ( Two of the four URLs are blog posts ( The graceful-degrade path — fetch failure falls back to the durable capability/thoroughness distinction with a visible note, never a guessed model name — is the correct posture and does not introduce a secondary injection surface. This surface is identical in character to the already-accepted Other surfaces — all clear
|
|
Claude finished @kyle-sexton's task in 2m 47s —— View job PR Review — feat(planning): interview recommends downstream model, effort, and advisor
SummaryThe PR is clean at HEAD. The two fixup-commit resolutions from the prior review cycle ( Two open Codex threads remain, which the author correctly classified as design-intent questions rather than clear bugs. My read of the current file state confirms they are real tensions — minor, non-blocking — and adds precision about what would resolve each. Findings1. "Downstream execution session" framing doesn't apply to terminal general sessions (low)
The design intent is defensible — a hard general decision can warrant a better-configured session for whatever comes next — but the preamble's "downstream execution session" is engineering-specific language that doesn't fit a terminal general session. The A one-line clarification to the preamble (e.g., "downstream execution session (or, for general decisions, the next session in which the decision is actioned)") would close this without changing any behavior. 2. Mid-task inverse direction reads as instruction to an executing agent (low)
However, the imperative phrasing ("surface", "recommend the upgrade") reads as instruction to an agent operating this context during execution. A model reading this in a future execution context (if this file were ever loaded in a different skill) would take it as an active instruction rather than relay-to-user text. Tightening the framing — e.g., "When handing off, advise the user to watch for these mid-task signals: if execution shows…" — removes the ambiguity without changing what the interview surfaces. Not a blocker given the file is only loaded at the interview boundary, but worth a follow-up tightening pass. 3. Graceful-degrade eval assertion not isolatable (observation, pre-existing)
Fixup-commit confirmationBoth major findings from earlier review rounds are resolved at HEAD (
Positive notes
|
Summary
Implements #231. The
interviewskill already reads task complexity and ambiguity to drive its rounds — this turns that read into a recommendation, at the stop/handoff boundary, for how the downstream execution session should be configured: model tier, effort level, and advisor pairing. The current model names and accepted pairings are researched dynamically at runtime from the official docs and never pinned in the skill, so the guidance does not rot as models and recommendations change.Fix
Root cause: the natural place to recommend a session's model/effort was doing complexity assessment but never surfacing a configuration recommendation. Added the behavior where the assessment already happens, following the sibling
draft-goal-conditionskill's established live-doc discipline (read the current official docs each run; hardcode nothing).skills/interview/SKILL.md— new## Session-config recommendation (model, effort, advisor)section (lean hub, points to the context spoke). Frontmatter/description unchanged (no new invocation trigger — this is an internal output of the existing flow).skills/interview/context/session-config.md(new spoke) — the detail:/model,/advisor, the effort setting.skills/interview/templates/checklist.md— Step 5 now includes the recommendation.skills/interview/evals/evals.json— new eval (id 9) exercising the behavior, including the graceful-degrade and never-pin expectations.0.22.2 → 0.23.0(minor — new capability) + CHANGELOG entry.Verification
(Both WARNs are pre-existing: SKILL.md soft-line-target 255>200, and the description's non-'Use when' phrasing — neither introduced by this change; all 5 base-ref trigger phrases preserved.)
Both live doc sources were confirmed reachable and current during authoring:
code.claude.com/docs/en/model-config.mdandcode.claude.com/docs/en/advisor.md(the latter documents the Sonnet main + Opus advisor pairing).Closes #231
Related
draft-goal-conditionskill (planning v0.22.0).