Repository navigation
feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651
Description
Refs: #7580
Refs: #7555
Decision: #7555 (comment)
Blocked by: #8816 (adds the app-control plugin and /app-control:drive, whose report.json this step reads).
Agent Brief
Type: Feature
Summary: After the deterministic tests pass, the verifier runs the project's own app-control skill against the PR head and folds what it finds into its verdict. Stack-neutral: the project names its skill; Docker is used only if that skill uses it.
Problem: The verifier judges a PR from its diff and text; nothing drives the running app the way a user would. The line that asked for this ("agent browser checks run after the deterministic tests, with no write token while PR code runs") sat in Phase 9, which closed with it unbuilt.
Current behavior:
.github/workflows/pr-verify.yml:3-27: runs onworkflow_runafterpr-require-checkscompletes; "No pull request code runs here"; job tokens can only read (:66-68,:384-386).- No app-control plugin exists yet; Phase 13 creates
/app-control:driveand the generatedcontrol-<app>skill shape.
Desired behavior:
- A setting in
docs/conventions/pr-pipeline.yamlnames the project's app-control skill. Unset: the step is skipped and the verdict says no app check ran. - When set: a separate job, after the deterministic checks pass, checks out the PR head and runs
/app-control:drive --unattended, which finds the project's control skill; the verdict job judges drive'sreport.json(drive itself never judges; D54). It holds no write token and no secret beyond the model credential the run needs, on a fresh GitHub-hosted runner, under ADR 0049's hardening. - Its findings go to the verdict job as data, framed as untrusted (
docs/conventions/untrusted-content/README.md); a blocking finding makes the verdict FAIL. - Nothing in the workflow names a stack. Docker appears only if the project's skill starts it.
Key interfaces:
.github/workflows/pr-verify.yml;.github/scripts/pr-verify/write-verdict.mjs.docs/conventions/pr-pipeline.yaml(new key) anddocs/conventions/pr-pipeline/README.md("Verifier verdict")./app-control:drive'sreport.jsoncontract from feat(app-control): add the app-control plugin with /app-control:drive #8816 (plugins/app-control/skills/drive/reference/evidence.md).
Done when:
- The diff adds the setting, the new job and the verdict input, and the job's
permissions:grant no write scope (read from the diff). - With the setting unset, the verdict notes that no app check ran (a
workflow.test.mjscase). CI'stest-nodepasses on the PR (a PR check; this run does not run it). - CI's
check-pluginspasses on the PR (a PR check; this run does not run it). - Security & Cost's pass on the final head is recorded on the PR (
.github/workflowschanges; owner rule 2026-10-20).
Out of scope:
- Building app-control itself (app-control: new plugin plus testing and debugging changes #7581).
- A stack-specific default (Docker or any other).
Origin:
- Verifier agents that run the app and click around like a user, then fix and repeat: hers (Lauren, stream ~53:10), adapted.
- Docker app stack plus browser checks in CI: the owner's (2026-10-16); moved to Phase 13 and made stack-neutral: ours, confirmed by the owner 2026-10-20.
Review: security (runs PR code in CI).
Do not run with: other pr-verify.yml changes.
Size Large, Complexity High, Uncertainty Knowable.
Slice of #7580 (Phase 13), filed from the owner's 2026-10-20 disposition round.
Activity