Skip to content

planning: eval coverage for interview lock-mode STOP-on-gap and auto-detect auto-guard (aihero lane 4) #2997

Description

@kyle-sexton

Context

Filed by course lane 4 (#2902). The lane's audit verdict: planning:interview's lock and auto-detect synthesize-directly paths are a LICENSED exception to the asset-rush failure mode, resting on four structural defenses — lock-invocation-as-confirmation, the STOP-on-gap rule, the auto-guard (a genuinely-user decision is never folded into the Brief silently; unattended it becomes a blocked register row with arbiter: USER-RESERVED), and the mechanical register gate. The first two of those are behavioral rules with no regression coverage — this item turns the audit's "defenses hold" from prose into a gate.

Proposed work

  • Eval case A (STOP-on-gap): a lock invocation over a task with a planted, genuinely-open decision (real tradeoffs, no codebase answer) — assert the skill HALTS and surfaces the gap rather than synthesizing a Brief that fudges it
  • Eval case B (auto-guard): an auto invocation whose context resolves all but one interactive design choice — assert the residue is asked (or, unattended, recorded blocked with arbiter: USER-RESERVED) and never silently captured as an assumption
  • Wire per the repo's eval conventions (evals:design / skill-quality:check validate-evals); planning plugin version bump + CHANGELOG

Acceptance criteria

  • Both cases exist and fail against a deliberately-broken variant (prove the rules bite: pass→fail→pass)
  • Cases run green against the current skill body
  • Lane 4 audit row updated with the eval pointer when merged

References

  • docs/upstream/aihero-core-lanes.md (lane 4 rows)
  • plugins/planning/skills/interview/SKILL.md (Action Router lock, Step 1.5 auto-guard, unattended ladder)

Metadata

Field Value
Category unspecified
Area planning, evals
Ecosystem unspecified

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: mediumReal value, no hard deadline; normal backlog flow.

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions