Repository navigation
fix(writing): be-concise no longer declines out-of-scope reshapes, and its semantic-diff threshold is too vague to fire #12846
Description
Refs: #8635
Summary
Three be-concise cases converted from skills/be-concise/evals/evals.json in #8635 fail on every run (Sonnet agent, Opus judge, 3 runs, with-arm, run-validity VALID):
| Case | Scores | Failing graders |
|---|---|---|
be-concise-declines-agent-facing-prose-routes-to-write-for-agents |
0.29, 0.43, 0.57 | expectation-1..4, names-write-for-agents |
be-concise-wall-of-text-routes-to-adhd-clarify |
0.375 x3 | expectation-1..4, names-adhd-clarify |
be-concise-long-tracker-comment-to-bluf |
0.78 x3 | expectation-5, semantic-diff-dispatched |
The legacy suite was never run against the current body. #8459 states that skills/be-concise/evals/evals.json "is unchanged and was not run".
Defect 1: the body routes but no longer declines
#8459 (58b75f3) reworded two Boundary entries:
**Does not reshape agent-facing instruction prose.** ... is docs-hygiene:write-for-agents when the docs-hygiene plugin is installed.became**Agent-facing instruction prose** (...): docs-hygiene:write-for-agents when the docs-hygiene plugin is installed.**Does not restructure a dense message without shortening it.** ... is adhd:clarify when the adhd plugin is enabled.became**Restructuring a dense message without shortening it** (...): adhd:clarify when the adhd plugin is enabled.
In the old wording the decline was unconditional and only the route was gated on the plugin. The new wording only routes. With the target plugin absent, nothing in the body tells the model to decline. In all 6 runs the model reshaped the text instead:
- Agent-facing excerpt: "I made the bullets and the opening summary line because you asked for them." Another run: "I followed your request for both anyway."
- Wall-of-text memo: every run produced a bulleted rewrite with word counts ("before about 450, after about 270").
The link to #8459 is UNVERIFIED: no run exists on the pre-#8459 body.
Fix: restore the unconditional "Does not ..." form for both entries in plugins/writing/skills/be-concise/SKILL.md (Boundary section), keeping the presence-gated route as the second sentence. Also align the docs-hygiene gates to "enabled", which #5985 standardized on; the body still says "installed".
Defect 2: "longer than a few paragraphs" does not fire on a five-paragraph, 422-word comment
SKILL.md "Guard the meaning" dispatches the semantic-diff subagent "for anything longer than a few paragraphs". The tracker fixture has 5 body paragraphs and 422 words. 0 of 3 runs dispatched. One run said: "I didn't run the subagent check because the text is short."
Fix: give the guard a threshold a model can check. For example, more than three paragraphs or more than about 250 words. Pick the number with an eval run, not by judgment alone.
Eval-side notes (not skill defects)
- The plugin eval loads only
writing, andclaude plugin eval --helphas no flag to load a second plugin. Soadhd:clarifyanddocs-hygiene:write-for-agentsare never present in the run. The graders that require naming those routes (names-write-for-agents,names-adhd-clarify, and adhd expectation-1 and expectation-2) test a branch the sandbox cannot load. Once Defect 1 is fixed, the body should still name the route with its gate. Re-run to see whether those graders pass with the plugin absent. - The run's init line names the subagent tool
Task, whilesemantic-diff-dispatchedmatchestool: 'Agent'. No trace under the run directory contains a subagent call, so whethertool_used: Agentmatches aTaskcall is UNVERIFIED. Check it once Defect 2 is fixed and a dispatch happens. The short-comment case'sno-subagentgrader has the same question. evals/fixtures/wall-of-text-migration-decisions.mdsets a 2026-10-02 deadline. That date is now past, and every run led with a date warning. Move it to a date far in the future, or use a relative date.
Acceptance
- After the body fix, the three cases score 1.0 on the with-arm, apart from any grader the absent plugin makes unreachable, and the other
be-concisecases do not regress.
🤖 Generated with Claude Code
Activity
github-actions commentedon Nov 3, 2026
The triage sweep dispatched intake triage for this issue (dispatch 2; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).
github-actions commentedon Nov 3, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Small
Complexity: Low
Uncertainty: Knowable
Flags: none
Blocked by: none
Understood: in
be-concise's SKILL.md, the two Boundary entries now only route and no longer decline. Restore the unconditional "Does not ..." wording and change the docs-hygiene gates from "installed" to "enabled". The "longer than a few paragraphs" semantic-diff guard also needs a threshold a model can check, and the fixture's past deadline date needs moving. The refinement comment's acceptance criteria cover this scope. The exact threshold number is an implementation choice, to be picked with an eval run. No duplicates found among open issues.Automated triage, applied by the lanes App.
melodic-automation-lanes commentedon Nov 3, 2026
Not admitted to the issue lane: no trusted-human acceptance criteria: a listed human must put the confirmed
- [ ]criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Nov 3, 2026
Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.
Intended outcome: Restore unconditional decline wording for the two Boundary entries, align docs-hygiene gates to "enabled", and replace "a few paragraphs" with a numeric threshold in plugins/writing/skills/be-concise/SKILL.md, plus move the fixture deadline date.
Acceptance criteria
External state: none
Dependencies
Related work
Sources read
Automated refinement by the lanes App. Intake triages the issue again with it.
github-actions commentedon Nov 3, 2026
github-actions commentedon Nov 3, 2026