session-flow/orchestrate: plugin-quality audit — decision criteria, defaults, handoff-relay convention

[kyle-sexton]

Goal

Behavioral audit of session-flow:orchestrate (route via /plugin-quality:audit): is it smart enough to be invoked in ANY conversation and do the right thing by default?

Audit focus (from interview 2026-07-25/26)

1. Decision criteria quality — what criteria drive its delegate/fan-out, model-routing, and calibrate-to-conditions imperatives? Are they stated, grounded, and current — or could they steer a session into wrong models / wrong orchestration shapes? (Operator has not used it recently; defaults must carry the load.)
2. Model-routing guidance — check against the user-scope model-routing lanes (subagent model: tiers, effort-first escalation) and current official model-config docs, fetched live.
3. Zone/context awareness — should its calibrate-to-conditions imperative consume the context-guard seam (presence-gated)? Cross-plugin rules per PLUGIN-PHILOSOPHY.md.
4. Handoff-relay convention (session-flow/workflow: end-of-phase continuation router (continue / clear / handoff / subagent / compact) #1476): worker writes own handoff at zone boundary, returns path only; parent spawns successor pointed at the file without reading it. Does orchestrate's brief guidance support this shape? Should it?
5. Standard plugin-quality dimensions: claims vs behavior, harness-claim currency, trigger discipline.

Outcome

plugin-quality findings + remediation items per its normal flow.

Criteria (landed)

• /plugin-quality:audit run against session-flow:orchestrate (repo HEAD d5513ca, all five focus areas)
• Findings filed per plugin-quality flow (2 handoff-inbox items; 8 findings: 1 HIGH, 5 MEDIUM, 2 LOW/NIT)
• Step-5 quality-gate review returned PASS-WITH-NOTES; F2.1 empirically confirmed live

Related

• session-flow/workflow: end-of-phase continuation router (continue / clear / handoff / subagent / compact) #1476 (convention definition), context-guard: zone-crossing hooks (advisory/enforcing) + window-class token bands #1475/context-guard: research non-interactive observability (headless + subagent context usage) #1478 (zone data availability).

[kyle-sexton]

This was generated by AI during triage.

Triage note. This item carried priority: medium but no routing outcome (no work-class, no role label, no native type) — an incomplete triage pass, not already-triaged output, so this completes it rather than re-triaging a decided item. Role labels resolved from .work-item-tracker.json config.role_labels: the binding has no role_labels key, so the defaults apply (agent-ready / needs-human) — flagging this per the taxonomy's loud-warning rule in case this repo intends a remap. Native Issue Type set to Task (org repo; no type: label exists here). No redundancy or rejected-concept match found; no existing plugin-quality audit of session-flow:orchestrate located.

Agent Brief

Type: Task
Summary: Run a /plugin-quality:audit pass over session-flow:orchestrate, focused on decision-criteria quality, model-routing guidance, zone/context awareness, and the handoff-relay convention.

Current behavior: session-flow:orchestrate has not had a behavioral audit checked against current harness docs and the other in-flight session-flow/context-guard conventions. Its decision criteria for delegate/fan-out, model-routing, and calibrate-to-conditions imperatives are unverified — the operator has not used it recently, so its defaults must be trustworthy without supervision.

Desired behavior: A /plugin-quality:audit pass that evaluates, and produces findings + remediation work items for, each of the following (fetch current official docs live for anything harness-behavior-related — never rely on training-data recall):

1. Decision criteria quality — are the delegate/fan-out, model-routing, and calibrate-to-conditions criteria stated, grounded, and current, or could they misroute a session into the wrong model tier or orchestration shape?
2. Model-routing guidance — check against the user-scope model-routing lanes (subagent model tiers, effort-first escalation) and current official model-configuration docs.
3. Zone/context awareness — should the calibrate-to-conditions imperative consume the context-guard presence-gated data seam? Evaluate per this repo's cross-plugin presence-gating rules (docs/PLUGIN-PHILOSOPHY.md).
4. Handoff-relay convention support — per the convention defined in session-flow/workflow: end-of-phase continuation router (continue / clear / handoff / subagent / compact) #1476 (worker writes its own handoff at a zone boundary and returns only the path; the parent spawns a successor pointed at that file without reading it itself), does orchestrate's brief guidance already support this shape? Should it?
5. Standard plugin-quality dimensions — claims vs. actual behavior, harness-claim currency, trigger discipline.

Acceptance criteria:

• /plugin-quality:audit completes against session-flow:orchestrate covering all five focus areas above.
• Every harness-behavior claim made in the audit is checked against a live-fetched official doc, cited by URL.
• Findings and any remediation items are filed through the normal plugin-quality workflow / work-item seam.
• This item is closed on delivery of the audit findings; no PR is expected from this item — the audit is the deliverable, and any code changes it surfaces are separate, independently-scoped follow-up items.

Out of scope:

• Do NOT modify session-flow:orchestrate's SKILL.md or any other file as part of this item — findings get filed as new work items, never applied inline here (this is what keeps the item C1/read-only).
• Do NOT resolve the open design questions in context-guard: zone-crossing hooks (advisory/enforcing) + window-class token bands #1475 or session-flow/workflow: end-of-phase continuation router (continue / clear / handoff / subagent / compact) #1476 — those are separate, already human-gated items; this audit may reference them but must not restate or re-decide their content.
• Do NOT build or wire the context-guard consumption described in focus area 3 — assess whether it should happen and file that as a follow-up if warranted.

[kyle-sexton]

[kyle-sexton]

This was generated by AI during work-loop execution.

Done (2026-07-26): Ran /plugin-quality:audit against session-flow:orchestrate (repo HEAD d5513ca1, plugin.json 0.17.10) covering all five requested focus areas. Per the resolved plugin-quality config (sink: markdown-dir), the full findings write-up landed as two items in the operator's cross-terminal handoff inbox (C:/Projects/melodic/handoff-inbox/) for a separate implementer session to pick up — no PR from this item, per its own acceptance criteria.

Items filed:

• 20260726-135918-session-flow-orchestrate-audit.md — the actual audit findings (8 findings: 1 HIGH, 5 MEDIUM, 2 LOW/NIT). Top finding (F2.1, HIGH): orchestrate's nested-subagent doc claims (context/sources.md + SKILL.md Tiered-delegation section) are stale and now factually wrong — nesting is off-by-default today, not "a configurable default of three" as currently written, independently re-verified twice against the live sub-agents doc plus confirmed as a reproducible live hazard (CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH unset everywhere checked on this machine). A sibling plugin (discovery) already has the correct current text in-repo, right now — in-repo SSOT drift. Also covers: two of imperative 7's four calibration factors (context pressure, rate-limit headroom) have no consumable signal despite presence-gated plugins for both being installed; the handoff-relay convention (session-flow/workflow: end-of-phase continuation router (continue / clear / handoff / subagent / compact) #1476) is not yet supported by orchestrate and imperative 4's current wording actively tensions against it; skill-quality:check PASS (0 errors, 1 warning: no Gotchas surface). Full detail, doc citations, and cheapest→most-ambitious remediations per finding are in the item.
• 20260726-135919-plugin-quality-audit-self-referential-process-findings.md — two process findings about plugin-quality:audit v0.1.2 itself, surfaced while running this audit: (1) its evidence-packet write contract assumes a dispatched agent/subagent can write findings.md-shaped files into the packet, which a harness-level report-file Write policy blocked twice in this run (worked around by renaming); (2) its Step 4 "interview the user" has no documented autonomous-invocation fallback (unlike e.g. work-items:work's own explicit clause), so this run improvised one and recorded the improvisation.

Verdict on the underlying question ("is orchestrate safe to invoke in any conversation and do the right thing by default?"): yes in the sense that matters most for a priming skill (side-effect-free, validates clean, export modes correctly tool-agnostic, low over-trigger risk) — but not fully trustworthy without qualification given F2.1's now-incorrect nested-subagent guidance.

Full evidence packet (evidence capture, contract lock, superseded findings draft) at ~/.claude/plugins/data/plugin-quality-melodic-software/evidence/d92258a9-b6d9-4555-83b5-90b8e576a35a/session-flow-orchestrate/20260726T134156Z/ on the executing machine.

[kyle-sexton]

This was generated by AI during work-loop execution.

Post-close correction. The dispatched step-5 review agent (quality-gate check on the write-up, mentioned as still in flight in the prior comment) returned after the handoff-inbox items were already filed and this issue was already closed. Verdict: PASS-WITH-NOTES — it independently re-verified nearly every citation in the audit and went further than this session did, dispatching a live subagent that actually attempted a nested spawn on this machine right now: the Agent tool was entirely absent from the nested subagent's tool list, empirically confirming F2.1 (the audit's HIGH finding) beyond doc-reading alone.

One real gap it caught: SKILL.md:131-135 co-cites the changelog page (not just the sub-agents doc) for "a configurable default of three," and only the sub-agents doc had been checked before filing. 20260726-135918-session-flow-orchestrate-audit.md has been updated in place (item still status: unclaimed, nothing claimed it yet) to add the live-reproduction evidence and reconcile the changelog citation rather than leave it unaddressed — see F2.1's "Empirically reproduced live" and "Changelog citation — reconciled, not dropped" subsections. No other correction was needed (a second item it flagged, a severity-tally inconsistency, existed only in the superseded evidence-packet draft, not in the filed item).