[kyle-sexton]

Closes #13758
Refs: #13757

session: App Driven PR Verify

Summary

Drive's mask post-check can now publish captures from this repository's CI capture. On main it cannot, for two reasons:

• No ## Mask in this repository's hub control skill. Without one, every capture is not-checked. Run 42235412941 recorded 12 captures and uploaded only the summary.
• Nothing for key auth to resolve. Under the CI key, the values: source has no account to name, and a source that yields nothing also leaves captures not-checked.

This PR adds two things to the control-skill contract and gives the hub its ## Mask:

• Hand-off. A hub's ## Mask may be the single line The driving tool's `## Mask`.
  • Drive reads the mask lines and the values: line from the control skill named on the hub's Driving tool: line.
  • It resolves that target from the project root of the hub copy it read, inside --trusted-root, never from the checkout under test.
  • A target that is missing, outside the root, without a ## Mask, or handing off again is unreadable, and every capture stays not-checked.
• none: values result. A cli/ source may print exactly one none: <reason> line and exit 0. That source then resolves to no values, and report.json records it under values_none. Any other output, empty output included, still fails closed.

Fix

• plugins/app-control/reference/control-skill.md "The Mask section": the hand-off, its resolution and trusted-root rule, and the none: result.
• plugins/app-control/skills/drive/reference/evidence.md "Mask post-check": follows the hand-off, accepts none: per source, and records values_none.
• plugins/app-control/skills/map/scripts/check-map.sh and its test: resolve the hand-off target and run mask-missing against it.
• Six new drive evals:
  • a hand-off reaches applied;
  • an unreadable target stays not-checked;
  • under --trusted-root, a target missing there stays not-checked and a decoy target in the checkout is not used;
  • a lone none: source is publishable;
  • none: beside a resolved source still scans the values;
  • none: plus an extra line stays not-checked.
• .claude/skills/control-claude-code-plugins/SKILL.md: ## Mask hands off to plugins/harness-ops/skills/control-claude-code/SKILL.md, whose 12 mask lines stay in one place.
• A changelog fragment (bump: minor).

Verification

• bash plugins/app-control/skills/map/scripts/check-map.test.sh: 74 passed, 0 failed (9 new cases).
• check-skill.sh passes on drive and map (0 errors), and check-evals-quality.sh passes on the drive evals.
• shellcheck, shfmt and markdownlint are clean, and all 37 drive evals parse.
• check-map.sh on the repository hub: clean.
• A copy of the hub with a screenshot capture resolved the harness-ops target. With the path broken, it reported mask-missing ... does not exist.

Remaining

• Merge hold: this PR changes skill bodies, so it carries do-not-merge until a valid before/after eval lands (hub rule R379/R381).
• The rest of feat(app-control): publish masked CI captures through a hub Mask hand-off and a none: values result #13757:
  • run-e2e follows the hand-off and accepts none: (testing plugin, next PR);
  • account.mjs prints none:: PR 13762.
• Live proof (feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364 rows 1 and 8): a CI capture that publishes a masked screenshot. It waits on PR_VERIFY_CLAUDE_API_KEY credit.

Related

• feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364
• feat(app-control): publish masked CI captures through a hub Mask hand-off and a none: values result #13757
• PR 13762

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	❌ Failed 2026-11-06T17:10:06.778239Z We couldn’t complete this review. Comment @codex review to try again.	5831350	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[github-actions]

claude-security-review reviewed e91c56e76652d42f5c9fe7f112c60475bd374b7f: read 7 of 7 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #13765, head e91c56e

I read all 7 changed files in full and found no security issues. Since this lane already reviewed an earlier head (5831350), this review reports blocking findings only (CRITICAL and IMPORTANT). Nothing from the earlier review carries over: it posted 0 findings.

critical-findings: 0

What I checked

1. The hub's new ## Mask hand-off. The hub, .claude/skills/control-claude-code-plugins/SKILL.md, is a trusted path. Its new ## Mask is just the hand-off line.

• The hub's Driving tool: line names plugins/harness-ops/skills/control-claude-code/SKILL.md.
• That target's ## Mask holds 12 mask lines and values: cli/account.mjs.
• So captures can now be published, but each one still has to pass drive's existing post-check:
  • the sidecar is scanned for every values: value and for the generic credential and PEM pattern;
  • only captures marked applied or none-declared are published;
  • a capture with a secrets_mark entry is never published.

2. Can a pull request under test supply its own masks or values? I checked whether code under test could weaken the check this way. It can't.

• The contract text in control-skill.md "The Mask section" and step 1 of evidence.md "Mask post-check" resolve the target against the copy of the hub that drive read. Under --trusted-root, the target's real path must be inside the trusted root.
• The target's values: sources run from the target's own cli/, so they also come from the trusted copy.
• A target that is missing, outside the root, has no ## Mask, or hands off again fails closed: every capture stays not-checked.
• Evals 34 and 38 cover this. Eval 38 is new at this head: the trusted target is missing and the checkout holds a decoy, and the expected result is that the decoy is not used.

3. The new none: <reason> values result. This is a narrow exception to the fail-closed rule.

• The source must be a cli/ source, exit 0, and print exactly one none: line with a non-empty reason.
• Empty output, a non-zero exit, or any extra line still leaves the source unresolved (eval 37).
• A mixed set of sources still scans the values that did resolve (eval 36).
• Only the trusted control skill's own cli/ command can produce none:, so text from a pull request or issue cannot reach it. Whether account.mjs prints it only under key auth is decided in PR 13762, not in this diff.

4. check-map.sh. This script is a read-only linter. It never runs anything the target names.

• It rejects any .. segment, and leading / is excluded by the path regex.
• It rejects a symlinked target file (-L) and checks the target folder's pwd -P against root_real.
• All variables are quoted, and file content goes only to grep, awk and sed, so there is no injection surface.

Other notes

• Trust controls: none changed. No change to the trusted-actor list, token scope, the kill switch, or merge holds.
• Plugin manifests: none changed.
• Instruction surface: the skill-body changes add a hand-off and an exception that is pinned by evals. Removing no-fallback or fail-closed wording would be a concern, but none was removed. The protected-class register was not consulted for this narrow addition.
• Embedded text: the PR context contains no directive aimed at this lane.

[github-actions]

claude-review: 1 inline finding(s) on 58313505edac8bf1ebe9b99bcb023ddfd5ab2ebd; the summary is in this lane's pull request comment.

claude-lane: claude-review

[github-actions]

claude-review reviewed e91c56e76652d42f5c9fe7f112c60475bd374b7f: read 7 of 7 changed file(s); 0 finding(s), 0 as inline comments.

Code review: PR #13765, mask hand-off and the none: values result (blocking only)

This lane already reviewed an earlier head (58313505), so this review reports blocking findings only. I read every hunk of all 7 changed files. I also read the unchanged parts of check-map.sh that the new code calls: the root and root_real resolution, unfenced, h2s and section_lines.

0 blocking findings.

Prior finding

• Eval 34 had no trusted-root decoy case (thread). The new eval 38 fixes this.
  • Eval 38, mask-hand-off-trusted-target-missing-checkout-decoy-not-checked, passes --trusted-root /srv/trusted/shop and leaves the trusted target missing. The checkout under test supplies a decoy tools/shop-driver/SKILL.md holding none: nothing to hide.
  • It expects not-checked, and checks that drive does NOT take the decoy as the mask in effect.
  • A drive run that falls back from the trusted root to the checkout now fails an eval.
  • Eval 34's own lines did not change, but the gap it was flagged for is now covered. I am not posting it again.

What checks out

• check-map.sh: resolve_mask_target handles each case separately:
  • a .. component, a symlink as the last part of the path, and a real directory outside root_real are rejected as leaving the root;
  • a missing file, a target with no ## Mask, and a target that hands off again each get their own reason;
  • a hub that is not a hand-off keeps mask_file as its own SKILL.md, so the existing mask-missing path runs unchanged.
• Tests: each branch has a test case in check-map.test.sh.
• Docs: evidence.md steps 1–5 match the contract in control-skill.md.
  • none: is judged one source at a time and fails closed on an extra line, empty output or a non-zero exit.
  • The new values_none field is listed in the report.json table.
• Closing issue feat(app-control): publish masked CI captures through a hub Mask hand-off and a none: values result - part 1 of 3 #13758, done-when:
  • drive evals cover the hand-off (33), the unreadable target (34, 38), a lone none: (35), mixed sources (36) and an extra line (37);
  • the hub gains ## Mask;
  • the test file has the hand-off fixture cases.
• Changelog: bump: minor fits a backward-compatible feature. No plugin.json or marketplace.json changed, so there are no manifest lines.

Notes

• I did not run the test results in the PR description (74 passed, shellcheck, check-skill.sh). They are the author's claims; this lane has no shell.
• The PR description still says "Five new drive evals". Six were added, and the decoy case it attributes to eval 34 is now eval 38. This is advisory, so it is not reported under blocking-only.

[kyle-sexton]

Held on do-not-merge only until a VALID no-loss before/after plugin-eval pair lands for this head (skill-body change, hub rule R379). The pair is running in the App Driven PR Verify session.

[kyle-sexton]

Eval pair: VALID, no loss. plugins/app-control, before 7c55519e and after 58313505, model and judge claude-opus-5-5, Claude Code 2.1.319, --scaffold --allow-tools Bash Write Edit --runs 3 (6 on two cases).

• Result: head minus base, paired over 48 cases: +0.00, 95% interval -0.02 to +0.03, non-inferior at margin 0.05 (noise-report.py --baseline --margin 0.05). Mean with-arm score 0.92.
• Validity: every counted case is VALID on both arms. The 36 desk cases pass run-validity.py on the desk. The laptop cases passed it on the laptop, per the hub's relay of the laptop lane verdicts.
• Same host per case: both arms of each case ran on one host with the same Claude Code version. Cases were split desk/laptop only to share load.
• Excluded, 1 case: skills-folder-plugin-keeps-project-source. It is excluded on both sides because it stayed INVALID after one rerun with --disallow-tools Glob Grep: runs reached into the plugin directory (Bash cd plus sed), which the sandbox denies.
• Reruns:
  • manifest-declared-plugin-command-read-from-disk, unpinned-checkout-stops and steps-include-kind-write-blocked replace earlier INVALID runs with clean reruns.
  • steps-finally-runs-after-failure has 6 extra runs per arm.
• Head drop read as noise: steps-finally-runs-after-failure (0.79 to 0.62).
  • Every failed run on both sides drops "finally": true on reload-billing: 1 of 9 base runs, 3 of 9 head runs. Fisher exact two-sided p = 0.58.
  • finally is defined in drive SKILL.md and evidence.md's steps row, and this diff changes neither.
• Measured head vs current head: measured at 58313505. git diff --stat 58313505..e91c56e7 -- plugins/app-control shows only skills/drive/evals/evals.json, the skill-creator format. claude plugin eval does not read that format: "the skill-creator plugin runs a similar comparison with its own evals/evals.json format, and neither tool reads the other's case files" (plugin-evals). Nothing under skills/drive/ references evals/.

Per-case scores (48 cases)

Case	Base	Head	Runs/arm	Host
animation-capture-app-not-launched	1.00	1.00	3	desk
animation-capture-no-tool-gap	1.00	1.00	3	desk
check-copied-recipe	1.00	1.00	3	desk
check-dead-index-line	1.00	1.00	3	desk
check-missing-not-driven-reason	1.00	1.00	3	desk
check-over-size-budget	1.00	1.00	3	desk
flat-legacy-feature-id-resolves	1.00	1.00	3	desk
grouped-feature-id-resolves	1.00	1.00	3	desk
grouped-folder-wins-over-flat	0.93	0.87	3	desk
integrated-app-composes	0.75	0.75	3	desk
manifest-declared-plugin-command-read-from-disk	0.83	0.83	3	laptop (rerun)
map-config-reports-source-layer	1.00	1.00	3	desk
map-converts-flat-control-skill	1.00	0.89	3	desk
map-discovery-before-questions	1.00	1.00	3	desk
map-failed-sign-in-stops-run	1.00	1.00	3	desk
map-feature-refresh-returns-to-start	1.00	1.00	3	desk
map-legacy-dirty-map-blocks	1.00	1.00	3	desk
map-legacy-flat-map-migrates	1.00	0.93	3	desk
map-lesson-every-feature	1.00	1.00	3	desk
map-lesson-not-a-feature	1.00	1.00	3	desk
map-lesson-one-feature	0.83	1.00	3	desk
map-maintainer-confines-edits	1.00	1.00	3	desk
map-prove-leaves-evidence-no-orphan	1.00	1.00	3	desk
map-refresh-ticket-branch-commits-there	1.00	1.00	3	desk
map-refreshes-each-control-skill	1.00	1.00	3	desk
map-reports-legacy-upkeep-branch	1.00	1.00	3	desk
map-source-wave-capped-at-six	0.67	0.83	3	desk
map-triage-doc-drift	1.00	1.00	3	desk
map-triage-driver-gap	0.50	0.83	3	desk
map-triage-driver-gap-unattended	1.00	1.00	3	desk
map-triage-product-regression	1.00	1.00	3	desk
map-unattended-safety-question-stops	1.00	1.00	3	desk
map-unlisted-recipient-stops-at-form	1.00	1.00	3	desk
map-writer-refuses-unsafe-location	1.00	1.00	3	desk
missing-other-checkout-stops-with-setting	0.92	0.75	3	desk
production-write-refused-by-default	0.67	0.50	3	desk
project-control-skill-with-hooks-read-as-data	1.00	0.87	3	desk
session-plugin-names-do-not-block-project-match	0.93	1.00	3	laptop
skills-folder-plugin-command-read-from-disk	0.83	0.92	3	laptop
steps-duplicate-id-stops	1.00	1.00	3	laptop
steps-finally-runs-after-failure	0.79	0.62	6	laptop (6 runs)
steps-include-kind-write-blocked	0.67	0.67	6	laptop (rerun, 6 runs)
steps-include-with-undeclared-stops	0.78	0.89	3	laptop
steps-optional-skipped	0.92	0.83	3	laptop
threads-capture-blocked-unattended	0.94	0.89	3	laptop
threads-capture-recheck-fails-interactive	0.75	0.79	3	laptop
unattended-production-write-off	0.80	0.80	3	laptop
unpinned-checkout-stops	0.44	0.56	3	laptop (rerun)

[melodic-automation-lanes]

The do-not-merge hold waited only on eval evidence (reason=eval), and a listed human posted a VALID no-loss eval-pair record for e91c56e76652d42f5c9fe7f112c60475bd374b7f: #13765 (comment). pr-merge removed do-not-merge. To hold again, add the label back with a new pr-hold reason line.