[kyle-sexton]

Closes #6605

Summary

Round 2 of the measured concise rewrite for the dometrain plugin, for the sync body only. Round 1 (#12116) shipped the grounding and setup bodies and held sync back, because its Opus result was decided by a grader defect (reports-exit-code missed "exited with code 2"). #12120 fixed that grader. On the fixed suite the round-1 sync text was measured again, both versions fresh, and it shows no loss on Opus 5.5 or Sonnet 5.5.

With this PR every #6605 part has a final result:

Skill	Part	Decision	Basis
grounding	body	after text (shipped in #12116)	round 1
setup	body	after text (shipped in #12116)	round 1
sync	body	after text ships	sy-check-report-only 1.000 → 1.000 on both models; every pass VALID; no grader defect
all three	description	unchanged	no change, so no decision

Fix

• sync/SKILL.md drops two sentences, the same text round 1 measured:

  • a flavor sentence ("There is one upstream dependency and no CLI to upgrade, only vendored prose to diff.");
  • the role-split sentence about disable-model-invocation: true, which repeats Boundaries bullet 1.

  61 → 58 lines, 392 → 353 words. Frontmatter, spokes, scripts and vendor/ are unchanged.

• Changelog fragment .changes/dometrain/, bump: patch.

Verification

• Protocol: rewrite-a-plugin runbook and the skill-criteria rewrite protocol, with a preregistration written before any run. Its decision rule is round 1's, unchanged: VALID on both models, after ≥ before − 0.05, no drop of a third or more. It also added two checks:

  • a defect rule: a grader defect on the deciding case means no ship and no close;
  • a leniency check: the round-1 and current reports-exit-code patterns are replayed on every reply, and any reply that passes only under the current pattern is read in full.

• Frozen suite: plugins/dometrain/evals git tree 03bb41f0bd (test(evals): fix eval-suite defects lane 3 measured in six plugins #12120's frozen digest), re-checked by file digest before every pass. Nothing under evals/ changed.

• Meaning diff: reused from round 1. Both texts are byte-identical to round 1's, and two fresh labelers each returned PASS with 2 INTENDED CUT and 0 loss.

• Runs: claude plugin eval (Claude Code 2.1.316), before and after launched together, --runs 5 --ablation none --case sy-check-report-only --allow-tools Bash Write Edit. Five runs because with one 6-grader case, a single miss moves the mean by 0.056 at 3 runs, more than the 0.05 margin.

  Case	Opus	Sonnet
  sy-check-report-only (with-arm mean)	1.000 → 1.000	1.000 → 1.000
  run-validity	VALID / VALID	VALID / VALID

  No grader failed in any of the 20 runs. The only run-validity warning was "skill fired", because sync is slash-invoked. There was no permission denial and no run error. The noise report reads "n too small to call", as expected for one case; it shows no case drop line.

• Leniency check: two Opus before replies pass only under the widened pattern ("exited with code 2 (unverifiable)"). Read in full, each states one exit code, so the grader is not lenient. Every other reply passed both patterns and states one code.

• Quota and network: no run called an mcp__ tool, WebFetch or WebSearch. The only network call was update.sh's GitHub fetch, which the sandbox refused, so every run took the exit-2 path on both versions. No Dometrain server is reachable from the eval sandbox.

• Host safety: a Bash-only probe ran before any grant was given. Windows drives were empty to the sandbox, the Windows binaries could not run, and WSL_INTEROP was unset.

• /skill-quality:check: 3 PASS on the branch, warnings 3/2/2, the same as round 1's in-repo before run (sync's before text is byte-identical to that run's). sync has 0 errors.

• /evals:validate (validate-cases.py) on plugins/dometrain/evals at the branch head: 0 FAIL. The WARN lines are the known grant notices.

• Totals: 5 passes (probe included), 1.64 USD list-price estimate, about 3 minutes of run time.

• gitleaks: 1 commit, no leaks.

Related

• #12116: round 1, grounding and setup bodies.
• #12120: the grader fixes this round measured on.
• #11833: the cases and probes.
• #12142: follow-up for the two round-1 behavior findings this PR does not carry.
  • On Sonnet, grounding did not fire on the plain JWT question (gr-covered-topic). This is a description-level gap.
  • On Opus, 4 of 10 gr-export-injection replies did not name the planted note.
• Still unmeasured and unchanged by this PR: anything that needs a live Dometrain server, and sync's drift path (exit 0 or 1), because the eval sandbox cannot reach raw.githubusercontent.com.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T23:07:11.620290Z	33372bf	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-review reviewed 33372bf847b1a767e9d698c19543d957947986f6: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no findings

I read every hunk of both changed files:

• plugins/dometrain/skills/sync/SKILL.md drops two sentences from Purpose:
  • The "one upstream dependency, no CLI to upgrade" flavor sentence.
  • The disable-model-invocation: true part of the role-split paragraph. The protection that sentence described still holds. The frontmatter still sets disable-model-invocation: true (SKILL.md#L5), and Boundaries bullet 1 still says "Never model-invocable" (SKILL.md#L53).
  • The remaining text about the report-only check and the maintainer-only --refresh-baseline is unchanged.
• .changes/dometrain/feat-6605-concise-sync-r2-5b8e1d47.md is a new changelog fragment with bump: patch. A wording-only change to a skill body fits a patch release, and the fragment accurately says what changed.

No plugin manifest is touched, so there are no manifest lines.

Closing issue #6605 "Done when"

• "The plugin's version or changelog entry is in the PR": met by the fragment.
• "Each model-invoked skill has a probe file: plugins/dometrain/probes/grounding.json": met. The file exists at this head (added in test(dometrain): add outcome eval cases and grounding trigger probes #11833) and the PR doesn't touch it.
• The PR body has the per-skill decision table, the before and after numbers, and the list of what was left unmeasured (sync's drift path and anything that needs a live Dometrain server).
• The /skill-quality:check and /evals:validate results are the author's claims. I did not run either, so they are unverified here.
• The eval numbers (sy-check-report-only 1.000 → 1.000 on Opus 5.5 and Sonnet 5.5) are also the author's claims and unverified. The sy-check-report-only case in plugins/dometrain/evals covers the behavior this text supports.

Spend

The PR reports eval passes that used prepaid API credit (about 1.64 USD list-price estimate). The PR adds no ongoing spend and starts no billed resource.

No embedded instructions were found in the PR context.

[github-actions]

claude-security-review reviewed 33372bf847b1a767e9d698c19543d957947986f6: read 2 of 2 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #12162 (feat(dometrain): concise sync body, round 2)

I found no security issues.

Scope. I read every hunk of both changed files in .claude-lane/pr.diff, plus the whole head plugins/dometrain/skills/sync/SKILL.md:

• .changes/dometrain/feat-6605-concise-sync-r2-5b8e1d47.md: a new changelog fragment with bump: patch. It is prose only.
• plugins/dometrain/skills/sync/SKILL.md: two sentences removed from the Purpose section.

No changed file matches a trusted-path glob. The PR changes no plugin manifest, workflow, script, hook or trust control.

Removed instruction text. The PR deletes part of a skill body, so I checked it against the deleted-instruction rules before deciding it had no security surface. I didn't read the instruction exception register (docs/contracts/instruction-exception-register/README.md), so I judged each removal by what breaks without it:

• "There is one upstream dependency and no CLI to upgrade, only vendored prose to diff." This only describes the skill. It forbids nothing.
• The sentence saying the skill is disable-model-invocation: true and runs only when a person types /dometrain:sync. The protection it described still holds in two places:
  • The frontmatter sets disable-model-invocation: true (SKILL.md#L5), which the harness enforces.
  • Boundaries bullet 1 still says "Never model-invocable" (SKILL.md#L53-L54).

The rules that stop the skill from running --refresh-baseline or editing vendor/SKILL.md and grounding/SKILL.md are unchanged (L14-L16, L33, L55-L58). No guardrail is lost.

PR context. The description and the issue comments contain no text that tells this review to pass the PR, skip a check or drop a finding.

Prior findings. This lane has posted nothing on this PR before (0 rounds with findings, cap 3, not past the cap).

Spend. The PR says its eval passes cost about 1.64 USD at list price, paid from prepaid API credit. Merging the change causes no charge.

critical-findings: 0