Concise skills rollout: visualization (2 skills, 2 model-invoked) #6595

Activity

1. kyle-sexton

   added a parent issue on Oct 7, 2026

   • Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476
2. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 7, 2026
3. github-actions commented on Oct 7, 2026

   

   github-actionsBot

   on Oct 7, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Medium
   Complexity: Medium
   Uncertainty: Known
   Flags: none
   Blocked by: none

   Understood: rewrite the two model-invoked skills in the visualization plugin (present, visualize) to the skill criteria in one draft PR, following the per-plugin runbook. The done-when list is explicit: skill-quality check, eval validation, a probe file per skill, a decision table in the PR body, unmeasured items named, and a version or changelog entry. The earlier hold was released by the owner's 2026-10-24 comment. I found no open questions and no likely duplicates.

   _{Automated triage, applied by the lanes App.}
4. github-actions

   added

   priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

   on Oct 7, 2026
5. kyle-sexton

   added the

   Task

   issue type on Oct 12, 2026
6. kyle-sexton

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   and removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 12, 2026
7. kyle-sexton commented on Oct 19, 2026

   

   kyle-sexton

   on Oct 19, 2026

   MemberAuthor

   More actions

   This was generated by AI during triage.

   Held: per the owner decision on #6476 (2026-10-18), this plugin is in the rollout middle group, which the owner decides after batch 2 reports its numbers. It waits for that decision.
8. kyle-sexton

   removed

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 24, 2026
9. kyle-sexton commented on Oct 24, 2026

   

   kyle-sexton

   on Oct 24, 2026

   MemberAuthor

   More actions

   Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.
10. github-actions commented on Oct 25, 2026

    

    github-actionsBot

    on Oct 25, 2026 – with GitHub Actions

    Contributor

    More actions

    The triage sweep dispatched intake triage for this issue (attempt 1 of 2).
11. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

    admission-deferred

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

    and removed

    admission-deferred

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

    on Oct 25, 2026
12. melodic-automation-lanes commented on Oct 25, 2026

    

    melodic-automation-lanesBot

    on Oct 25, 2026

    Contributor

    More actions

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Why, in the run's own words:

    The rewrite protocol (/playbooks:skill-authoring skill-criteria.md ＃＃ Measurement step 6, and reference/rewrite-a-plugin.md steps 1, 2, 5, 6, 8) ships only parts that were measured: it needs /skill-quality:check before and after, /evals:validate, paid /evals:plugin-eval passes on each target model, /skill-quality:check measure-invocation, and /docs-hygiene:compress compare. This CI lane loads only the work-items plugin, and its Bash runs only the lane wrappers (no claude CLI, no bash on repo scripts), so none of those can run here. Rewriting either skill without them would ship unmeasured text, which the protocol forbids, and the PR's per-skill decision table needs numbers this lane cannot produce. Done instead: step 3, the frozen probe sets for both model-invoked skills, written blind by fresh agents from the before descriptions and committed before any rewrite. A run that can execute the eval tooling (an attended session, or a lane with the evals and skill-quality plugins and the claude CLI) should continue from this branch.

    What the run says it did not do:

    • Skill rewrite of present and visualize (runbook step 4): not started, because the protocol ships only measured parts and measurement cannot run in this lane
    • /skill-quality:check plugins/visualization before and after: the skill-quality plugin is not loaded and its script cannot run under this lane's Bash allowlist
    • /evals:validate on the visualization eval suite, and the case-design, calibration and smoke pass (runbook step 2): the evals plugin is not loaded here
    • Paid eval passes before and after on each target model, plus the measure-invocation held-out trigger run (runbook step 5): no claude CLI or eval tooling in this lane
    • /docs-hygiene:compress compare meaning diff with two labelers (runbook step 6): plugin not loaded
    • Per-skill decision table with before and after numbers, and the list of unmeasured skills or cases, in the PR body: no numbers exist yet
    • bash plugins/skill-quality/scripts/measure-invocation.sh validate plugins/visualization/probes: cannot run here (Bash allows only the lane wrappers); CI runs it for every plugins/*/probes directory, left to the PR's CI
    • new-changelog-fragment wrapper refused a detached HEAD, so the fragment was written by hand in the same format; check-changelog-fragments accepts it

    Partial work is on the branch work-resolve-issue/issue-6595-40190516272-1. A retry continues from it while it is not behind the default branch, and otherwise starts fresh from the default branch; delete the branch first to always start fresh.

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40190516272

    To retry, remove needs-human and add agent-ready again.
13. kyle-sexton

    marked this as blocked on Oct 27, 2026

    • feat(ci): validate, gate and summarize dispatched plugin eval runs #9704
14. kyle-sexton commented on Oct 27, 2026

    

    kyle-sexton

    on Oct 27, 2026

    MemberAuthor

    More actions

    This was generated by AI.

    Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start claude or claude plugin eval (see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.

    Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off, and the issue moves to host-run with status: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.
15. kyle-sexton

    added

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.A maintainer confirmed the item is real and in scope; ready to be worked.

    and removed

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

    on Oct 27, 2026
16. kyle-sexton

    self-assigned this

    on Oct 31, 2026
17. kyle-sexton commented on Oct 31, 2026

    

    kyle-sexton

    on Oct 31, 2026

    Last edited by kyle-sexton

    MemberAuthor

    More actions

    No description provided.
18. kyle-sexton

    closed this as completedin #11623on Oct 31, 2026
19. kyle-sexton commented on Oct 31, 2026

    

    kyle-sexton

    on Oct 31, 2026

    MemberAuthor

    More actions

    Result: landed in #11623 (merged 2026-10-31T16:47:23Z).

    • Shipped: 6 regex-graded outcome eval cases under plugins/visualization/evals/ and frozen trigger probes probes/present.json and probes/visualize.json.
    • No skill text changed. Both bodies and the visualize description failed the margin on Opus. The present description tied on held-out trigger counts, but every trigger pass was judged INVALID, and skill-criteria.md Measurement step 4 reads a number only from a valid run, so its before text stays.
    • Re-measure and grader fixes: test(visualization): fix the frozen suite's graders and re-measure present and visualize with VALID runs #11628 (validity of trigger passes: fix(evals): let trigger-measurement probe runs pass run-validity #11618).
    • Evals: 16 measured passes plus 2 smoke passes, run one at a time; the decision table is in the PR body.