Skip to content

Epic: consolidate the planning chain #12646

Closed
22 of 22 issues completed
Closed

Description

@kyle-sexton

Refs #6917

Consolidates the planning chain: one stage order, one decision level per stage, one vocabulary, one view configuration on ADR 0075, one fallback chain, one chrome, live review pages, and presence-gated plugin offers. Sub-issues are the slices; this epic is never agent-ready.

Brief

TLDR

  • One stage order in /session-flow:workflow: interview first, then design, plan, decompose, then the autonomous lanes; the quick-change on-ramp stays.
  • Each stage decides one level: behaviour in the Brief, interfaces and contracts in design, files and symbols in the plan.
  • One vocabulary: rung, tier, ladder, overlay, surface and contract each name one thing.
  • One view configuration on ADR 0075's layers, one fallback chain, one shared chrome, live review pages with approval in the conversation.
  • Presence-gated guidance: a one-line offer of the installed surfaces and planners, and fresh, consented optional-plugin installs.

Problem

  • Gap: planning surfaces disagree. The workflow's stage list contradicts its own prose (plugins/session-flow/skills/workflow/SKILL.md:77-90); the interview, plan and brainstorm render views through different settings (surface vs the medium cascade); the medium setting still reads a file ADR 0075 retired (docs/contracts/rendered-views/README.md:802-804; ADR 0075:50, :128); one word names two things in the contract (rung :328 vs :806; tier :37 vs :822; surface :802); the interview pages do not use the shared chrome (prior explore EXPLORE-summary.md).
  • Who / how many: the owner (human-in-the-loop stages) and every marketplace consumer of the planning plugins; count unknown.
  • Frequency: every planning session.
  • Trend: unknown.
  • Cost: confusion at the human gate (round 1: owner could not tell view from medium); duplicated look and settings to maintain.
  • Evidence: strong (file:line reads above, verified explore EXPLORE.md).

Intent

"The interview skill and the planning skill are very, very important. They're very important to the human-in-the-loop steps ... that stuff needs to be very, very, very, very good. Because everything else after that should be, for the most part, automated through our agent-ready or software factory kind of deal." (owner, 2026-11-02)

Goal

A person starting work meets one consistent planning chain: the interview runs first and calls research, exploration, prototypes or a PRD only when a question needs them; each later stage decides its own level and hands a markdown record to the next; every page those stages show looks and behaves the same, falls back visibly when a medium is unavailable, and keeps approval in the conversation; the user is told which surfaces, planners and optional plugins are available without being stopped to choose.

  • Intent (why, in the user's words): "all of our plugins, planning, and all that stuff need to be consolidated ... I just want to make sure that we support all those things and then we gracefully degrade."
  • Vision (where this heads): "interview first, then planning, and all of those different sequential steps"; plugins ship Artifacts as the default view; the owner's machines use local HTML, later Cloudflare-tunnel shareable pages.
  • When goals conflict, prefer: none stated.

Work size

  • Appetite: not assessed.
  • Size: large: workflow, interview, design, plan, rendered-views contract, shared chrome, config resolver, publish gate, and a repository-wide vocabulary pass.
  • Uncertainty: low on what (every decision answered); moderate on how (mechanisms deferred to design and plan).
  • Split signals: separable slices per decision area (order, levels, vocabulary, config, fallback chain, chrome, live review, offers); one epic, Refs Epic: improve the planning chain from html-plan, HumanLayer rpi and Pocock sources #6917.

Constraints

  • Skills reference each other only by public contract (plugin namespace, skill name, public arguments), never internals (owner, 2026-11-02).
  • Markdown is the record; HTML, Artifacts and tunnel pages are views (ADR 0025; Epic: improve the planning chain from html-plan, HumanLayer rpi and Pocock sources #6917 C2).
  • Plugins ship the Artifact view as the default where the Artifact tool exists; in this epic that covers brainstorm, design and plan views only; the owner's machines use local HTML via enableArtifact false (owner, 2026-11-02; scope Q32).
  • No team rendered-views.yaml in org repos for now (Q31).
  • html-plan may be copied, integrated or referenced as an optional dependency that degrades gracefully, never required; comments built into our chrome (owner, 2026-11-02, replacing Epic: improve the planning chain from html-plan, HumanLayer rpi and Pocock sources #6917 Q2; prior Q13).
  • The shipped Artifact default is verified in CI with posted evidence, not on the owner's machines (owner, prior Q14); out of scope (owner, 2026-11-02).
  • Team standards: design-for-change (add a variant by addition, docs/standards/design-for-change.md:26), reference-dont-duplicate, simpler-code.
  • Quality requirement: Performance: rendering or watching a page never blocks the conversation; no numeric target.
  • Quality requirement: Reliability: every view lane falls back to the next reachable medium and says why, never a dead file path (docs/contracts/rendered-views/README.md:361-362).
  • Quality requirement: Security: no new path off the machine; existing paths keep their controls: an Artifact view publishes only when the publish gate passes and otherwise falls back to a local file (docs/contracts/rendered-views/README.md:383-386); a hosted page also goes to the private host for a non-public repository or a page with a machine path or hostname, and is refused on a credential-shaped line (docs/contracts/rendered-views/README.md:452-460); the shared session keeps its own controls; page input is untrusted data.
  • Quality requirement: Maintainability: each shared file has one canonical copy, drift-checked in CI (ADR 0019).
  • Quality requirement: Usability: the accessibility floor holds: contrast pairings, focus visibility, color-scheme and reduced-motion behavior, keyboard reach (docs/contracts/rendered-views/README.md:727-729); the theme follows the system setting by default.
  • Quality requirement: Scalability: does not apply (one user, one session per page).
  • Quality requirement: Observability: each view lane states the medium used and the deciding fact.

Acceptance criteria

  • AC1: /session-flow:workflow lists one stage order, interview first, and its list and prose agree (Q3).
  • AC2: The interview, design and plan skills each state the level they decide: behaviour, interfaces and contracts, files and symbols (Q4).
  • AC3: A search of the whole repository, excluding dated records (docs/adr/, docs/upstream/, CHANGELOG files) and the synced docs/standards/, finds none of these words used in their retired senses (each match is read in context): "rung", "default ladder", "overlay" in their old senses, "tier" for precedence, "surface" for a config file, "contract" for the Brief; the workflow stage reads "Interview" (Q5-Q12).
  • AC4: medium resolves through ADR 0075 precedence (plugin default, userConfig, docs/conventions/rendered-views.yaml, docs/conventions/rendered-views.local.yaml); neither ~/.claude/rendered-views.md nor .claude/rendered-views*.md is read as a settings layer (Q14; ADR 0075:167-169). Amended 2026-11-03 (owner, see AC14): a retired file is not a settings layer, but its terminal or file value still keeps the view local.
  • AC5: Setting medium never changes the interview's surface (Q15).
  • AC6: The contract records brainstorm, design and plan views as a named Artifact exception (Q16). CI evidence of the Artifact default is out of scope (owner, 2026-11-02: "CI verification evidence should be out").
  • AC7: The contract holds one fallback chain, ordered published Artifact, local file, terminal, with "hosted" read as Artifact except for PR digests (docs/contracts/rendered-views/README.md:366-368, :448-451); every view lane names the medium it used and the fact that decided it (Q17).
  • AC8: Our interview pages and planning views (not a third-party plugin's own pages) use the one canonical chrome, with a system/light/dark toggle and the comment UI, and no team or user theme overrides (Q18, Q36).
  • AC9: On a review page (a Claude-interactive view of a brainstorm, design or plan record), on the file and Artifact mediums, a choice redraws its example; a lock reaches the session and is written to the markdown record; approval is taken only from the conversation; a missed page event is picked up at the next message (Q19, Q33). If design finds no Artifact channel, an Artifact review page says so and review happens in the conversation; the terminal reviews in the conversation.
  • AC10: A plugin is offered only if listed, not renamed, and last committed within 6 months, with its date shown when older than 3 months; it installs at user scope only after a yes and points to /reload-plugins; no offer appears in an unattended run (Q20, Q21, Q34).
  • AC11: One epic exists, Refs Epic: improve the planning chain from html-plan, HumanLayer rpi and Pocock sources #6917, slices as its sub-issues; the epic carries no agent-ready label (Q2, Q22).
  • AC12: Between human-in-the-loop stages the workflow suggests a handoff only when context-guard warns (Q23).
  • AC13: With the surface not set by the user and more than one surface, the shared session or an optional planner (a presence-gated planning plugin offered under AC10; html-plan today) available, the interview names the default and the alternatives in one line and proceeds; an installed html-plan is named as an alternative and is never required (Q25, Q35).
  • AC14: A view lane that finds a retired rendered-views file (~/.claude/rendered-views.md, .claude/rendered-views.md or .claude/rendered-views.local.md) reads it only to name its value and where to move it (plugin userConfig for the user file, the matching docs/conventions file for a repo file), once per view-lane plugin per project (Q29, Q50 reconfirmed at planning, owner 2026-11-02), never applies the value, and names a team value that selects a publishing medium as ignored (Q29, Q50). Amended 2026-11-03 (owner, in session, verbatim: "Go with recommended, but those docs cloud locations, all that, should be targeting or the normative state should be going to docs/convention/whatever."): a retired file's terminal or file value keeps the view local; a publishing value there stays ignored; the move notice names docs/conventions as the destination (docs/conventions/rendered-views.local.yaml first for the user file, the plugin userConfig as the per-user alternative). Reason: a security finding on refactor(adhd): resolve the clarify view medium through the shared resolver #12966 showed that ignoring a local value could publish an Artifact for a user who had set output to stay local.
  • AC15: Adding a medium, a surface or a shared-session variant touches its own registration and renderer, with no edit to the skills that use it (Q24).

Captured assumptions

  • Quality bar: production
  • Variation intent: second variant = known; existing structure: four mediums (terminal, file, artifact, hosted) through the cascade and default ladder, two surfaces (terminal, page) plus the shared session; upstream: Epic: improve the planning chain from html-plan, HumanLayer rpi and Pocock sources #6917 workstreams W3-W7
  • Acceptance-criteria coverage: unwanted-behaviour and state-driven cases covered by AC7 (unreachable medium) and AC9 (missed page event); owner accepted with no addition.
  • Mechanisms (sync script, redraw attribute, page lock event) are inputs to design and plan from RESEARCH.md items 6-7: revisit if design finds a simpler mechanism.
  • Owner's machines: the review plugin's userConfig sets medium hosted; every other view lane local (Q30): revisit if the owner wants PR digests local.
  • No team rendered-views.yaml in org repos (Q31): revisit if other people work in org repos with Artifacts on.

Out-of-scope

Deferred questions

  • Q38: answered at planning (owner, 2026-11-02): option (b), CI renders the view under the Artifact page rules; then taken out of scope by the owner.
  • Q39: Which plugin setting the carry-over notice names, and where the shown-once state lives; defer until planning; arbiter: /planning:plan
  • Q40: Where the offerable-plugin list lives, how a rename is detected, and whose last commit counts; defer until planning; arbiter: /planning:plan
  • Q41: Canonical chrome home (plugins/visualization/reference/html-chrome.html or lib/); defer until planning; arbiter: /planning:plan
  • Q43: answered at design (owner, 2026-11-02): an Artifact review page offers a "Copy choices as prompt" control; the owner pastes it into the conversation; no new path off the machine (design/design-threads.md T4).
  • Q42: Which context-guard zone counts as the warning that suggests a handoff; defer until planning; arbiter: /planning:plan

Load-bearing assumptions

Assumption Source Falsifying check Response if false
The sync script, redraw binding and page-lock event are the mechanisms; revisit if a simpler mechanism appears Brief Captured assumptions Phase 3 Sanity Check (redraw test, no network call) Minor
The owner's machines set medium hosted in the review plugin userConfig only; revisit if the owner wants PR digests local Brief Captured assumptions (Q30) Phase 1 Sanity Check (review lane resolves hosted from userConfig) Minor
No team rendered-views.yaml in org repos; revisit if others work in org repos with Artifacts on Brief Captured assumptions (Q31) Phase 1 Sanity Check (org code search finds no docs/conventions/rendered-views.yaml) Minor
The cascade's file reader and schema check can be reused per scope without changing resolve() for its four existing callers plan Phase 1 Sanity Check (existing config-cascade tests pass) Moderate: add a per-layer export to the cascade and bump its carriers

Execution shape: per-item PRs

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 3, 2026
  2. github-actions commented on Nov 3, 2026

    @github-actions
    Contributor

    Proposed work class: work-class: structural
    Basis: automated intake triage; the lanes App applied this class.

    Size: Large
    Complexity: High
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    Latest decision, quoted:

    Decision recorded.

    Owner, 2026-11-05 (lane 2 session, answering Q2 with "go w/ recommendations"): option (a). No slice is reopened. The --runs 10 Loss results for planning:interview and planning:plan cannot be traced to any one slice, because the controls show these suites do not exercise the medium text. #12646 stays open on these criteria, which join its existing list:

    • Root-cause interview-negative-recommendation-yes-accepts (with-arm 0.97 at BEFORE 8a40f13010 → 0.67 at AFTER 2b63ab6c79, --runs 10) on current main, and fix the cause in the interview skill or the case, with a rerun at --runs 10 showing the case back at or above 0.90.
    • adhd:clarify, architecture:improve and planning:design are recorded as not measurable in the claude plugin eval sandbox. Their deciding cases need reads outside the workspace (the retired ~/.claude/rendered-views.md, or the plugin's own copy), which the sandbox denies. Recorded in the final report above.

    The before/after eval criterion itself is met: the run is done and its report is linked above.

    #12646 (comment)

    This decision settles the open questions. The issue is a large epic with sub-issues as slices, and it stays non-agent-ready as its brief states. No duplicates found.

    Automated triage, applied by the lanes App.

  3. added
    priority: mediumReal value, no hard deadline; normal backlog flow.
    and removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 3, 2026
  4. added
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    work-class: structuralRefactors, migrations, contract changes; cross-cutting and hard to reverse.
    on Nov 3, 2026
  5. added
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Nov 3, 2026
  6. melodic-automation-lanes commented on Nov 3, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: its done-when asks for what an unattended lane run cannot perform: item 1 (eval: "One claude plugin eval before/after run over the view-lane skills' existing e..."); recommended split: keep the other items here as the admissible issue and move these to a child issue for the host queue (host-run). Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  7. melodic-automation-lanes commented on Nov 3, 2026

    @melodic-automation-lanes
    Contributor

    The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

  8. 65 remaining items

  9. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 5, 2026
  10. added
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    and removed
    status: readyTriaged, unblocked, and fully specified; eligible to pick up.
    on Nov 5, 2026
  11. added
    status: needs-decisionAwaiting a human or maintainer judgment call.
    and removed
    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.
    on Nov 5, 2026
  12. melodic-automation-lanes commented on Nov 5, 2026

    @melodic-automation-lanes
    Contributor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Keep the epic open until the interview-negative-recommendation-yes-accepts case is root-caused and fixed, with a --runs 10 rerun at or above 0.90; every other slice is merged.

    Related work

    Open questions

    1. Can an unattended lane run finish the remaining criterion, given that proving it needs a claude plugin eval at --runs 10 that previous runs did only on the host laptop (lane 2)?
      RECOMMENDED: No. Route the remaining item to the host eval lane as a check a person or laptop lane runs, and keep the epic non-agent-ready as its brief says. Basis: judgment
    2. Should the epic body get a task list holding the remaining criteria, so intake stops marking it thin?
      RECOMMENDED: Yes. List three items: root-cause and fix interview-negative-recommendation-yes-accepts (rerun at --runs 10 at or above 0.90); Q39-Q42 resolved at planning; the epic closes when all sub-issues are closed. Basis: judgment
    3. Is the interview-negative-recommendation-yes-accepts fix a separate sub-issue on an interview skill or case change, rather than part of the epic?
      RECOMMENDED: Yes. File it as its own slice. The case's prior fix is the wording rule in plugins/planning/CHANGELOG.md:2144, so the cause may be a regression in that rule or in the case's prompt at plugins/planning/evals/interview-negative-recommendation-yes-accepts/prompt.md. Basis: plugins/planning/CHANGELOG.md:2144
    4. Do Q39-Q42 (notice setting, offerable-plugin list, chrome home, context-guard zone) still block anything now that their slices are closed?
      RECOMMENDED: No. Their arbiter is /planning:plan and the slices that needed them (#​12660, #​12663 and others) are closed completed. Mark them resolved in the epic, or confirm the closed slices answered each. Basis: #​12660

    Sources read

    • /home/runner/work/claude-code-plugins/claude-code-plugins/refine-data/context.json: epic body, owner Decision recorded of 2026-11-05 (option a, adds the root-cause criterion), the eval final report, and the closed slice candidates
    • plugins/planning/CHANGELOG.md:2144: the interview wording rule that made interview-negative-recommendation-yes-accepts pass earlier
    • plugins/planning/evals/interview-negative-recommendation-yes-accepts/prompt.md:3: the eval case exists in the repository

    To answer, post a comment that starts with this line:

    **Decision recorded.** <answer>
    

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

  13. kyle-sexton commented on Nov 6, 2026

    @kyle-sexton
    MemberAuthor

    Root cause of interview-negative-recommendation-yes-accepts (0.97 → 0.67): the eval environment, not the skill text. Agent finding, lane 2.

    Cause. The AFTER interview skill runs node surfaces/registry.mjs resolve before round 1. My planning launch scripts did not put the real node binary first on PATH (lane rule R71, already applied to discovery under addendum 2), so inside the eval sandbox node was the mise shim, which fails with mise ERROR node is not a valid shim. BEFORE never calls node.

    • planning --runs 10 confirmation: the shim error appears in 160 of 360 AFTER with-arm runs (all interview cases) and 0 of 360 BEFORE; in this case, 10 of 10 AFTER runs.
    • planning --runs 3 pass: 86/192 AFTER and 83/192 control with-arm runs, 0 BEFORE.
    • run-validity.py passed those runs because a failed Bash command is neither a denial nor a run error.

    Test (preregistered before any run: ≥ 0.90 means environment, ≤ 0.80 means skill text). The unchanged AFTER copy, this case only, --runs 10, real node first on PATH:

    • Pass 1: all 10 runs stopped on a weekly usage limit before any model turn; rerun (environmental).
    • Pass 2: INVALID (1 permission denial: a run copied the plugin's checklist template from outside the workspace). I read its score, 0.83 (inside the undecided band), before running validity, against lane rule R72. It does not count.
    • Pass 3: VALID (run-validity.py 66e4081), 0 shim hits: 0.93. Its 2 failed runs fail only the yes-accepts judge, the same polar-wording shape as BEFORE's 1 of 10.
    • Disclosed: passes 2 and 3 set a fixed PATH without the shim directory rather than <node>:$PATH.

    Closing pass on current main (d23b140, R71 PATH form, preregistered as Addendum B): 0 shim hits; run-validity.py 66e4081 says INVALID on 4 denials, all the same cp of skills/interview/templates/checklist.md from the plugin copy into the workspace. Under the preregistered clause, a denied run is voided and the call is made on the rest: 6 valid runs, 1.00 (all 10 runs: 0.93). The criterion "back at or above 0.90 with-arm at --runs 10 on current main" is met.

    That copy denial is a sandbox property: dontAsk denies reads of the plugin copy outside the workspace. It is not a skill defect, but it will keep voiding interview runs in this suite.

    Verdict changes to the final report.

    • planning:interview: the Loss is withdrawn. Its AFTER and control runs were confounded; BEFORE was not. The suite can't call it either way until it is re-run with R71 applied.
    • planning:plan: the Loss stands. Its cases had 0 shim hits.
    • Other plugins: shim hits are about equal on every side (prototype 12/42 each, education 3/51 each, work-items 2–4/30 each). architecture:improve is AFTER-heavy (8/40 BEFORE, 21/40 AFTER), but its verdict was already no loss, so it holds.
    • The owner's Q2 decision ("reopen no slice", comment) is unchanged; withdrawing this Loss only removes a reason to reopen.

    Fix. Environmental only: no skill text changes. Lane rule R71 now covers any plugin whose skills or scripts call node, not only node hooks.

  14. kyle-sexton commented on Nov 6, 2026

    @kyle-sexton
    MemberAuthor

    Closing as completed. Every criterion is met:

    • Before/after eval over the view-lane skills, run and reported: final report.
    • Owner Q2 decision applied, no slice reopened: decision.
    • Root cause of interview-negative-recommendation-yes-accepts found (eval environment: the node shim), and the case is back at or above 0.90 with-arm at --runs 10 on current main: root cause and closing pass. That comment also withdraws the planning:interview Loss.
  15. kyle-sexton commented on Nov 6, 2026

    @kyle-sexton
    MemberAuthor

    Disclosure: the traces behind the root-cause comment were deleted after it was posted.

    After #12646 closed, lane 2 deleted its eval sandboxes under its cleanup rule, which removed all 2,472 kept run directories, including every trace.jsonl. The score files remain, but their tracePath entries now point at nothing. As a result:

    • The shim-error counts (160 of 360 AFTER interview runs, 0 BEFORE, and the per-plugin counts) can't be re-checked from traces. The counting script and its full output survive only in the lane-2 session transcript on melo-lap-001; the table is copied to a local record.
    • Other Epic: consolidate the planning chain #12646 verdicts can't be re-judged against the shim error from traces. Doing so would take new runs.

    The lane's cleanup rule is changed: each kept run's trace.jsonl is archived beside its result JSON before a sandbox is deleted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

priority: mediumReal value, no hard deadline; normal backlog flow.work-class: structuralRefactors, migrations, contract changes; cross-cutting and hard to reverse.

Fields

Work status

needs-owner-decision

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions