Skip to content

feat(discovery): make research restatements cite claim IDs and check their labels - #9965

Merged
melodic-automation-lanes[bot] merged 3 commits into
mainfrom
work-resolve-issue/issue-6070-40182269191-1
Oct 25, 2026
Merged

melodic-automation-lanes[bot] merged 3 commits into
mainfrom
work-resolve-issue/issue-6070-40182269191-1

Conversation

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor

Closes #6070

Summary

Opened by the work-resolve-issue lane. The run's own summary:

Research restatements (Summary, Settled and handoff lines, catalog cells, sidecar abstracts) now cite a claim ID instead of restating the claim, and any line naming a HIGH (single source), MEDIUM or LOW claim carries that label on the same line. artifact-shape.md defined no claim ID, so this adds an id: field to the sidecar claims header. A new Python-only checker, check-restatement-labels.py, reports every line that names a labeled claim's ID without its label. Per the recorded decision it is advisory: the parent runs it after the acceptance gate and hands what it reports to the verifier. Part 3 (the rework cap) is not built here: the issue body scopes it to #8072, and SKILL.md's 'Effort, source breadth' already caps rework at one round, which replaces the decision comment's N=5. SKILL.md:88 now points at that bound.

Not verified

  • The new check-restatement-labels.test.sh, contract.test.sh and check-source-applicability.test.sh were not executed: this lane's Bash runs only the named wrappers, and check-affected runs without --tests. Their behavior was hand-traced by the reviewer instead. Left to the PR's CI test lane.
  • The issue's 'PR CI runs contract.test.sh and check-evals-quality.sh and both pass' is a CI check; check-evals-quality passed here, and contract.test.sh is left to the PR's CI.
  • Ruff lint on the new .py was not confirmed: the pinned ruff wrapper cannot be invoked from this lane, and check-affected's output does not name which linters ran. Left to CI's lint lane.
  • No permission allow rule was added for check-restatement-labels.py in /discovery:setup's rule list or parent-contract.md's operator-setup block. Until one is added, the advisory check prompts on each run, and a declined prompt is recorded as 'not run' per dispatch.md. The issue did not ask for this,
  • The deep tier's workflow path (research-sweep.js, deep-tier.md step 5) does not write id: into the headers it generates, so the restatement check finds ids=0 on those artifacts until it does.
  • The decision comment's part 3 (a 5-round rework cap in dispatch.md) is not built: the issue body now scopes it out as covered by #8072, whose one-round bound already sits in SKILL.md 'Effort, source breadth'.

Fix

  • plugins/discovery/scripts/check-restatement-labels.py: new advisory checker. It reads each sidecar's claims: header with check-source-applicability.py's parser (loaded via importlib) and collects claims that have an id: and a HIGH (single source), MEDIUM or LOW label. It then flags each line of the
  • plugins/discovery/scripts/check-restatement-labels.test.sh: new black-box test. Covers the fixture pair (exit 1 with six named violations, exit 0 corrected), label matching (bare pointer, HIGH alone, LOW/BELOW), whole-token ID matching (M1 vs M10, S2 vs S2a, P1-P3), table rows, sidecar bodies and ab
  • plugins/discovery/tests/fixtures/restatement-labels/unlabeled/RESEARCH.md, RESEARCH-perf.md: fixture slice modeled on the issue's residuals, where restatements drop the single-source and MEDIUM labels.
  • plugins/discovery/tests/fixtures/restatement-labels/corrected/RESEARCH.md, RESEARCH-perf.md: the same slice with every restatement shrunk to a labeled pointer.
  • plugins/discovery/skills/research/context/artifact-shape.md: adds id: to the sidecar claims schema plus its grammar paragraph. Adds the 'Restatements cite; they do not restate' rule, extending the verbatim-abstract rule to Summary, Settled/handoff and catalog cells, and names the checker.
  • plugins/discovery/skills/research/context/dispatch.md: new section 'The restatement check is advisory' with the command, how to read exit 0/1/2, and why no exit halts the run. Adds a 'Brief it on restatements' step that turns each reported line into a verifier Restatements: line (graded under row 12
  • plugins/discovery/skills/research/SKILL.md: line 88's 'no limit on iterations for run-owned rows' becomes 'rework bounds: "Effort, source breadth"', two bytes shorter so the 20,000-byte anchor slice holds. The header carry-forward line names id.
  • plugins/discovery/skills/research/context/discipline.md: the single-source flag paragraph points at the claim-ID citation and the checker.
  • plugins/discovery/agents/researcher.md: mirrors the cite-not-restate rule, including writing an id: on every claim and shrinking a failed restatement to a pointer.
  • plugins/discovery/agents/research-verifier.md: adds an optional Restatements: dispatch field and a row 12 restatement check (row 7 for MEDIUM/LOW Gap claims shown as settled).
  • plugins/discovery/scripts/contract.test.sh: section 24 pins the rule, the id: schema line, the advisory gate text, the verifier and researcher mirrors, the SKILL.md:88 pointer, the absence of 'no limit on iterations', and that the checker ships.
  • .changes/discovery/fix-6070-restatement-labels-6e7a0b31.md: minor-bump changelog fragment. Written by hand because new-changelog-fragment refuses a detached HEAD.

Verification

The runner ran each check from the base copy against 92938a3:

  • scripts/check-changelog-fragments.sh --check: passed

The run reports:

  • .lane/checks/bin/check-changelog-fragments: 'All 2 changelog fragment(s) under .changes/ are valid.'
  • .lane/checks/bin/check-affected: '11 passed, 5 skipped over 14 changed files'. The first run failed shellcheck SC2016 on a backticked single-quoted string in check-restatement-labels.test.sh; that was fixed and the rerun passed.
  • .lane/verify/bin/check-evals-quality plugins/discovery/skills/research/evals/evals.json: PASS, with 2 warnings on cases 11 and 17 that this change does not touch.
  • .lane/verify/bin/cant-fail-scan --file plugins/discovery/scripts/check-restatement-labels.test.sh --check: PASS, no gating findings.
  • Fresh-context review subagent over the whole diff: it hand-traced the checker against both fixtures and every test case and found them correct, and all section-24 contract assertions matching. It found doc contradictions: MEDIUM/LOW restatements graded under row 12, 'beside the ID' vs whole-line mat
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40182269191

Left out: the items under Not verified.

Related

#6070

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 898665b358586a8345cf09e0cdee890c4fb519f7: 0 finding(s), 0 as inline comments.

Incremental security review of plugins/discovery/skills/setup/evals/evals.json (since e5183af): no security issues found.

The only change is to one eval expectation string. It now says the printed allow rules cover "all four gate scripts" instead of "all three", and it names the four checkers. It is test data only. It runs no code, grants no permission, and changes no userConfig key or manifest field. It also doesn't delete, narrow or soften a standing instruction. The expectations around it are unchanged: no wildcard on the version segment, ~/.claude/settings.json is not written, and the operator must re-paste the rules after an update.

I did not read the setup skill's own text or the gate scripts to check that they really print four rules. That is a correctness question for the code-review lane, not a security one.

No trusted-path file was touched in this increment. The PR context file was not read, so there was no embedded text to report.

manifest fields: not hit by this increment.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 92938a3c11df6d33721be80203e94a91e2d07d58; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 898665b358586a8345cf09e0cdee890c4fb519f7: 0 finding(s), 0 as inline comments.

Incremental review of plugins/discovery/skills/setup/evals/evals.json (e5183af to 898665b): no findings.

The change edits one expectation in the check-prints-gate-allow-rules-for-operator case. It now says the printed rules cover "all four gate scripts (dispatch-artifact, coverage, source-applicability and restatement-label checkers)" instead of "all three gate scripts".

  • Matches the skill. plugins/discovery/skills/setup/SKILL.md lines 71-95 list eight rules, four scripts in quoted and unquoted forms: check-dispatch-artifact.sh, check-coverage-complete.sh, check-source-applicability.py and check-restatement-labels.py. The expectation's wording matches that list.
  • No stale count left. A grep of the setup evals for "three", "six", "eight" and "gate" finds no other expectation that still counts three scripts. The only "three" is an unrelated case about allowed explore_output values.
  • Closes a gap the PR flagged. The PR body's "Not verified" section says no allow rule was added for check-restatement-labels.py. SKILL.md and this eval now cover it.

I read the files only and ran no tests or evals. The JSON edit is a string change inside an existing array element, so it can't affect structure.

No userConfig or manifest change falls in this increment. No cost or critical-security hold applies.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

The unattended merge path armed native auto-merge at head 92938a3c11df6d33721be80203e94a91e2d07d58.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 1 of max 4 on e5183af.

@melodic-automation-lanes melodic-automation-lanes Bot added the needs-human Human-in-the-loop required; autonomous sessions must not resolve items carrying this. label Oct 25, 2026
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

The pr-unblock lane stopped on this pull request and needs a person: a fix the lane recorded did not reach the branch, so none of its review replies was posted.

  • Reason: feedback-not-landed
  • Resume key: e5183af3e9cc2420c29d8bc70a43446771fce26a (the head the lane stopped on)

To resume, deal with the reason above, remove the needs-human label (and do-not-merge, if set), then dispatch the pr-unblock workflow with pr-number=9965 and activity=address-feedback:

gh workflow run pr-unblock.yml -R melodic-software/claude-code-plugins -f pr-number=9965 -f activity=address-feedback

… eval

The setup eval's expected output already names eight rules across four
checkers; its expectation still said three, so a run that printed only
the original six rules would pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton kyle-sexton removed the needs-human Human-in-the-loop required; autonomous sessions must not resolve items carrying this. label Oct 25, 2026
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

The unattended merge path armed native auto-merge at head 898665b358586a8345cf09e0cdee890c4fb519f7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(discovery): stop restatements from failing research verification round after round

1 participant