Harness: run and apply for the review Intent, and the model holds no credential that can write

Part of the epic "Epic: slice 1, first approval. A Release on next reviews a pull request to this repo, and the machine user's approval stands".

Why. The first commit of harness/ and roles/: run and apply for the review Intent (§2.3 steps 7 to 13). Forty-six rows and eight lessons wait on harness/; nine on roles/. FR-10, "the model process holds no credential that can write", is held by nothing until this exists.

Done when

harness/task.py builds task/1 from a Decision and the facts, every text labelled by its trust class, saying what was left out and how much (L-56); harness/engine.py runs the pinned CLI (L-35) on the factory-run environment with tools disallowed, hands it result/1 with references resolved and the Harness's fields removed, and writes result/1, or a failed one with result_invalid for an answer that does not validate (Schemas for Task, Result, Effect and Disposition #19 ruling 9).
harness/apply.py on factory-apply asks the Kernel again with its own Run left out (I-38), re-reads the item and refuses if held, paused, closed or changed (I-21), refuses a Result whose run or release is not the Task's, an Effect the Task does not permit, a duplicate id, a Finding posted other than once or a non-gating one on a thread, a Disposition with neither half or both or a half its Role may not make, and a push to a protected path or a Lens's input with no model review (Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139, closed here); submits the review as a comment on the admitted commit, never an approval; mints a token narrowed to the requested Effects through the identity port; reads each Effect back; writes applied/1 and outcome/1.
roles/pr-review.json validates against role/1, names evals/pr-review, and §5.4's mechanical checks run over it in tools/check_authority.py (one owner per category; Effects within the App's ceiling; an Intent of class gate has an eval set).
tests/test_harness.py drives both jobs against a stand-in CLI and a fake effects port (as Trial scripts: P-13's script hardened before it runs again #178 does for the trial script), including: no job holds both a model credential and a write credential (L-43, held by the workflow's environments and tools/check_workflows.py); every refusal above; a claim is written only from what was read back (L-41). Every row waiting on harness/ and roles/ that this holds is re-pointed.
No Run has a shell or a network tool until P-26 passes (Trial P-26: the model CLI's sandbox hides the parent's environment and limits connections #188, Design: the CLI's sandbox limits connections but passes the parent's environment through (P-26 failed); the run job relies on the scrub and --tools "" #287): the run job starts the CLI with --tools "" (P-13 at 2.1.306 shows structured output still works), sets CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1, keeps the model credential only under CLAUDE_CODE_OAUTH_TOKEN and holds no other secret in its environment; tools/check_workflows.py holds all four. (Added 2026-10-21 by the orchestration session, owed by Trial P-26: the model CLI's sandbox hides the parent's environment and limits connections #188.)

Needs. The two ports, the eval set, #19 and #28 on main. The keys are stored; tests read recorded responses only.

Design. §2.2, §2.3, §3.5 to §3.8, §3.11, §4.9; decisions 0004, 0005, 0029, 0033, and #19's record; L-14, L-35, L-41, L-43, L-56.

Working rulings (overturnable in review)

Solution Architect, delegated. On main at 95b6adb. Read from:

the issue, and the two it names, "Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed" (Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139) and the model Lens's own inputs (A model Lens's inputs: its manifest and its prompt #140);
the slice-1 epic's Order ("Then: the Controller (lane b). run and apply (lane s). The workflows", waiting on the ports and the eval set) and its rule that the first story in a reserved directory re-points every row waiting on it;
design §2.3 (steps 7 to 13), §3.5 (Task and Result), §3.6 (Effects, the re-read, the two bindings), §3.7 (a raised Finding, the refusal of a non-gating one on a thread), §3.8 (a Disposition's two halves), §3.10 (applied.json, outcome.json, the act Run), §3.11 (the Role manifest), §4.3, §4.9 (the threats, the credentials table) and §5.1;
decisions 0004 (a job is a credential boundary; six rows waiting on harness/), 0005 (each Role writes through its own App), 0020 (a change to a Lens's inputs is reviewed by a model Lens), 0029 (an uncovered criterion has not reported), 0033 (the model credential is, for now, the Maintainer's own subscription; slice 1's CI credential reopens it), 0035 (each key lives in the environment of the one job that holds it), 0038 (a Result names no one; apply writes the Finding) and 0039 (the envelope);
schemas/task.schema.json, result, effect, disposition, applied, outcome, role and finding, as Schemas: Task, Result, Effect and Disposition (#19) #210 and Schemas: the Role manifest, the pass record, the feed, and one envelope for all (#28) #214 merged them; config/apps.json (the reader App on factory-run, pr-review on factory-apply); tools/check_authority.py; evals/pr-review/cases.json (the review Intent's set, count: 0);
trials/P-13.json and tools/trial_p13_cli.sh as Trial scripts: P-13's script hardened before it runs again #242, Hook tests: a budget measures the hook, not the machine #257 and Trial scripts: a reason is the Task's one code, every check message is hidden, and the after-Task version's exit is read #263 left them (the pinned CLI, --json-schema, structured_output, the allowlist of what is kept, --tools "" not yet run against the real CLI), and tools/trial_as_app.sh (an App's JWT signed by openssl from the key's path);
lessons L-14, L-35, L-41, L-43, L-56 and the 55 rows and lessons waiting on harness/ and 10 on roles/;
the facts, records and Controller rulings on Facts port: one snapshot of a pull request from GitHub, with its completeness flags #191, Records port: the Factory's own run records, unidentified Runs and the cache #192 and Controller: one pass reads, decides, records, dispatches, writes the view and raises the reconciliation alert #193.

What this story decides. What the two jobs that carry a Run are, as code: what run gives the model and takes back, and what apply checks, writes and records. It is the first commit of harness/ and roles/, and the first place FR-10, "the model process holds no credential that can write", is held by code rather than waited on.

1. Three pull requests, in order.

PR	Holds	Module	Waits on
1	The `run` job: the Task from a Decision and the facts, the CLI's call, the Result	`harness/task.py`, `harness/engine.py`, `harness/run.py`	The facts port's PR 1 (#191), for the facts it reads
2	The `apply` job: the re-read, every refusal, the token's mint, the Effects and their read-back, `applied/1` and `outcome/1`	`harness/apply.py`, `adapters/identity.py`	PR 1; the records port (#192), for the Result artifact; the effects port's first commit (#193 PR 2), or its fake
3	The Role: `roles/pr-review.json`, its prompt, and §5.4's checks in `tools/check_authority.py`; the job boundary in `tools/check_workflows.py`	`roles/`, two checks	PR 1; "A model Lens's inputs: its manifest and its prompt" (#140), before it or in it

Why: run holds the model credential and apply the write credentials; reviewing them apart is reviewing the boundary FR-10 draws. Each PR is testable with no I/O beyond a stand-in, as #178 tested the trial script.

2. PR 1: the Task. harness/task.py builds task/1 from the pass record's Decision and the item's facts, and nothing else (§3.5: "a Run started any other way finds no Decision and does nothing").

Each text is labelled by its trust class from who wrote it, read from the facts and the configuration, never from the text: maintainer for the Maintainer, role for a Factory App, advisory for a third-party reviewer listed in the configuration, outside for anyone else.
A Task says what was left out of it and how much (L-56). task/1 has no field for that today, so it gains an optional omitted: for each kind of text cut, the count cut and the characters cut, in codes and numbers. An optional field keeps the major version (0038: "a field added later is optional, or the major version goes up"). An example and a counter-example come with it.
limits, tier and permitted_effects come from the Role's manifest for the Intent; release from the Release that runs; content_id and item from the facts.

3. PR 1: the Engine. harness/engine.py runs the CLI as tools/trial_p13_cli.sh does, and reuses what #242, #257 and #263 hardened there rather than writing it again:

the pinned version (L-35), checked before and after the call, its exit read;
an environment of exactly the named variables, and no key from the job's environment but the one the CLI needs;
--tools "": no tool at all, so a Task that holds outside or advisory text has no shell and no network tool (I-35, 0004's last row), and the review Intent needs none;
--json-schema with result/1 as the CLI is handed it: references resolved, the Harness's fields (release, usage) removed, status narrowed to completed, incomplete and refused, effects narrowed to the Task's permitted_effects (§3.5);
of the CLI's answer, only structured_output and the fields usage needs, by allowlist; the model's prose is never kept or logged.
An answer whose structured_output does not validate is not a Result: the Harness writes the whole Result itself, failed, reason result_invalid, empty lists and usage (0038). usage is filled from the CLI's own answer; the model and the session are named before the call, so both are known even when the CLI never answers.

4. PR 1: what run uploads. result.json with its sha256, and nothing else from the model. The log carries no field of the Result (§3.10: "nothing is handed from job to job through the log").

5. PR 2: the re-read and the refusals. harness/apply.py, on factory-apply, takes the Result by the artifact id and digest run reported, never by name (§3.6). Before any Effect, in this order, it writes nothing and ends the Run when:

the Kernel, asked again with this Run left out, does not admit this Run's Intent on this content (I-38);
the item is held, paused, closed, or its content_id is no longer the Task's: the Run ends superseded (I-21);
the Result's run or release is not the Task's;
it asks for an Effect type the Task does not permit, or names an Effect id twice;
a raised Finding is posted by other than exactly one Effect, or a Finding that does not gate (a minor or info one of any category but security) is posted on a thread (§3.7);
a Disposition gives neither half or both, or the half this Run's Role may not make, or resolves a security thread on declined or deferred (§3.8);
a submit_review would be anything but a comment on the commit the Decision admitted (§3.6).
Each refusal is a reason code in applied/1 and in outcome/1, whose fault is item for a Result the Role wrote and superseded for a change under the Run.

6. PR 2: the token. adapters/identity.py mints an installation token for the one repo the Run is about, narrowed to the permissions the requested Effects need, as config/apps.json's needs names them.

Its JWT is signed by openssl, as tools/trial_as_app.sh signs it: the standard library cannot sign RS256, and the repo installs nothing. The key is written to a file of mode 0600 in the runner's temporary directory, read only by openssl, and removed in a finally.
The token is held in memory, never written, never logged, and revoked when apply ends.
It is the identity port's: Facts port: one snapshot of a pull request from GitHub, with its completeness flags #191 ruling 9 named the mint for this story.

7. PR 2: the Effects and the record. Each Effect is written through the effects port in effect/1's order, recovered by its key where one is already applied (§3.6), and read back. applied/1 lists each with its state (applied, already_applied, refused, failed), the node id read back and a reason code, and each Finding posted, written whole as finding/1 from what was read back (0038). A claim is only ever what was read back (L-41). outcome/1 is the last step's.

8. PR 3: the Role. roles/pr-review.json validates against role/1: kind reviewer, owner of the model-judged categories (§5.1), its App combsmith-sf-pr-review, its Intent review-change of budget class gate naming evals/pr-review. Its prompt lives beside it in roles/. tools/check_authority.py gains §5.4's three checks over every manifest: one owner per category, every permitted Effect within its App's ceiling in config/apps.json, and an Intent of class gate names an eval set that exists. That the review Intent may serve only once its set has cases stays the cutover's (tests/test_evals.py::test_the_review_intents_eval_set_has_its_cases, not held yet, #183).

9. PR 3: the job boundary. tools/check_workflows.py holds, for any workflow: no job names both an environment holding a model credential and one holding a write credential, and no job that holds a write credential checks out a Work item's branch (L-43; 0004's first two rows waiting on harness/). The Factory's own workflows are the workflows story's (#196); this PR adds the rule and counter-examples, so #196's files are held by it from their first commit.

10. What waits.

Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139, a Role's push to a protected path or a Lens's input that no model Lens reviewed: no slice-1 Intent can ask for a push, since effect/1 has no push_commits until the fix Intent (§3.6: "the other types come with the Intents that need them"). The refusal waits for that Intent; see the rulings on Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139.
A comment reaching the next Run (L-14's last clause) waits on the fix Intent, which reads threads to answer them.
The eval job with its read token is reserved by name only (The review Intent's held-out eval set, from the proof of concept's merged-then-reverted pull requests #183); it waits on the workflows story.
The live call. Every test uses a stand-in CLI and a fake effects port. The first live Run is the workflows story's and the Release's.

11. Tests and mutants. tests/test_harness.py drives both jobs against a stand-in CLI and a fake effects port, as tests/test_tools.py::TrialScript drives the trial script: the environment the CLI is given, --tools "", a Result that does not validate becoming result_invalid, every refusal of ruling 5 with its code, the token narrowed and never written, and a claim written only from what was read back. Each rule gets a mutant naming its test, checked as the shards run them and by the capped filtered run on each path. Every row waiting on harness/ and roles/ that a PR holds is re-pointed in that PR; the rest name the narrower file that will hold them.

12. Record. One, numbered at enqueue with PR 1: "The model is given a Task and no tool, and apply writes only what it reads back". Its table holds rulings 2 to 9. PR 2 and PR 3 add rows to it.

13. Gates. Export verify; a fresh Claude review and a security pass on each PR, the security pass reading the Engine's environment (PR 1) and the token's mint (PR 2) first. AGY on Argon, since this is the boundary FR-10 rests on and PR 2 asks the Kernel again. "Muse: not run (Maintainer, 2026-10-20: Kernel/hook only)". The capped filtered mutant run on each new path.

14. Out of scope. The workflows and their environments (#196). The approving act, a Run with no Engine (#195). The effects port's real writes (#193 PR 2). The author Intents and their bundle, preflight and push_commits. FR-12's injection suite (slice 2).

Needs the Maintainer? Not for building it. Three points are his:

The model credential in CI (cost and risk). 0033 says "slice 1's CI credential reopens this decision". The run job's secret on factory-run is either a token from his own subscription (claude setup-token) or a metered key under FR-73's cap. Nothing in this story uses either; the first live Run does. The decision should be made, and 0033 reopened, before Workflows: the frozen entry shim, the poller, the Run's three jobs and the nudge, each held to its shape #196 runs a Run.
--tools "" with --json-schema (risk). P-13 proved structured output with the CLI's default tools. That it still works with none is unverified until the next real run of tools/trial_p13_cli.sh (Trial scripts: what Muse's later rounds asked of trial_p13_cli.sh #247's fourth item waits on the same run). If it does not, the fallback is --disallowedTools naming every built-in tool, which the issue's own wording allows.
Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139 and A model Lens's inputs: its manifest and its prompt #140 (scope). Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139 cannot be tested in slice 1 and should move to the fix Intent's story; A model Lens's inputs: its manifest and its prompt #140 becomes live the moment roles/pr-review.json exists, and should land before PR 3 or in it. See their rulings.

Working rulings: the Maintainer's answers (2026-10-21)

The orchestrator relayed the Maintainer's answers to the points the rulings above left to him. The rulings file is updated to match.

The model credential in CI. The Maintainer's own subscription, by a token from claude setup-token under his login, stored as the run job's environment secret on factory-run. A metered API key under FR-73's cap is 0033's fallback. This story's record carries it, and reopens 0033 as that decision asked (ruling 12).
--tools "" with --json-schema. Joined to "Trial scripts: what Muse's later rounds asked of trial_p13_cli.sh" (issue 247), whose fourth item waits on the same real P-13 run at his terminal. Until then the Engine is built and tested against the stand-in; the fallback, if structured output does not hold with no tools, is --disallowedTools naming every built-in tool.
The App's JWT (ruling 6). The key reaches openssl through a pipe, as -sign /dev/fd/N, and is never written to a file; tried with a throwaway key on 2026-10-21, it signs. A file of mode 0600 removed in a finally is the fallback only where a runner's openssl cannot read a descriptor.
Harness: apply refuses a Role's push to a protected path or a Lens's input that no model Lens has reviewed #139 moves to slice 2 and this story does not close it. A model Lens's inputs: its manifest and its prompt #140 is a slice-1 Kernel story, re-scoped on its own issue, landing before PR 3; if it lands before PR 1, every PR here needs the Maintainer's approval, since each changes a model Lens's input.

The re-pin's fix round: 0033's row names no version, the test keeps t…

Ruling 4, the CLI's call: `--tools ""` with `--json-schema`, settled (2026-10-21)

The real P-13 run at the CLI's 2.1.306 shows that --tools "" leaves --json-schema working:

the structured output validated;
permission_denials was empty;
stop_reason was tool_use, because the CLI delivers structured output through a tool of its own, which --tools "" does not remove.

So the run job can disable every tool and still get its Result, and the --disallowedTools fallback is not needed. The fixture is trials/P-13.v2-1-306.json, landing in the re-pin pull request.

Working rulings, amended: the run job's environment (2026-10-21)

P-26 at the CLI's 2.1.306 failed. The CLI's sandbox limits connections and writes, but it hands a tool the parent's environment. Only CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1 removes names on its own list, CLAUDE_CODE_OAUTH_TOKEN among them. Filed as "Design: the CLI's sandbox limits connections but passes the parent's environment through (P-26 failed); the run job relies on the scrub and --tools """ (issue 287).

Ruling 3 (the Engine) and ruling 9 (the job boundary) gain these rules, and the record carries them as rows with ruling 12's. They bind from PR 1:

The scrub. The run job sets CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1 explicitly, in the environment the Engine gives the CLI. It is never left to a default.
One credential, under one name. The model credential sits only under CLAUDE_CODE_OAUTH_TOKEN, the name the scrub removes.
No other secret. The job's environment holds no secret but that one.
No tool on untrusted text. A Run whose Task holds outside or advisory text runs with --tools "". P-13 at 2.1.306 shows structured output still works, so the review Intent runs so always.
Held by the check. tools/check_workflows.py holds the scrub variable, and the absence of any other secret, in the run job, with counter-examples. It lands with ruling 9's job boundary in PR 3, and PR 1's Engine test holds rules 1, 2 and 4 in the environment it builds.

The rewording of design §4.9, and the Linux /proc/<pid>/environ trial, are issue 287's own small pull request.

Trial P-26 (#188) failed at 2.1.306; its fixture and design rows are on trials/p-26-sandbox. The model CLI's sandbox limited where a tool connected, but it passed the parent's environment to the tool.

The CLI's scrub, CLAUDE_CODE_SUBPROCESS_ENV_SCRUB=1, took out the one listed credential name that was tried.
The model credential's own name, a Linux runner's /proc and the CLI's configuration files were not tried.

#188's "if no" asks that this story's Done when gain one line, until P-26 passes:

No shell and no network tool for any Run, until P-26 passes.

The Maintainer or the story's owner adds it; this comment does not edit the issue.

Harness: the fix round of #194 PR 1, after the Claude review, the sec…

Harness: one description for harness/ on the list (#194, PR 1)

Harness: the run job reads and writes only inside the directory it ru…

Harness: #194's record takes 0050, by the orchestrator's order (#194,…

Harness: config/harness.json is pr-review's own input, and its pin is…

Working rulings, amended: the change reaches the model, and the credential is the org's (2026-10-22)

The session building the eval job (#309) found a gap: task/1 has no field for the change, and the model runs with --tools "", so neither a live review Run nor an eval run can see what it judges.

Ruling 2 gains the diff. For a pull request, the Task carries the change as one text, the first:

Id: diff:<the content id's hex>.
Trust: outside, whoever pushed it, since the change is the text a review judges and an injection can arrive in it.
Source: the last pusher.
Its own budget, config/harness.json's texts.diff_characters, counted escaped as the other texts are. Up to it the diff is kept whole; over it, it is cut at a file's boundary, never inside one. The Task's omitted says, for kind diff, how many files were left out and how many characters.
No new field: task/1 keeps its major version, and the Engine hands the diff to the model as data in the Task, like every text.

The run job is given the diff's bytes, and builds no Task unless their sha256 is the content id the Decision admitted. That is the hash the facts port takes of the same bytes (decision 0044), so a Run can only be given the content it was admitted for. A pull request with no diff, or one that is not text, gets no Task; an issue has none.

The credential is the org's (the Maintainer, 2026-10-22). The Claude credential is an org-level secret. The run job reads it from the org, under CLAUDE_CODE_OAUTH_TOKEN, not from the factory-run environment. Ruling 9's check in tools/check_workflows.py holds that only the run step names it. This replaces the 2026-10-21 wording "the run job's environment secret on factory-run".

Built in "Harness: the Task, the Engine and the run job (#194, PR 1)" (#304), with its record.

Harness: a pull request's Task carries its change, held to the conten…

Navigation Menu

Repository navigation

Harness: run and apply for the review Intent, and the model holds no credential that can write #194

Working rulings (overturnable in review)

Working rulings: the Maintainer's answers (2026-10-21)

Ruling 4, the CLI's call: `--tools ""` with `--json-schema`, settled (2026-10-21)

Working rulings, amended: the run job's environment (2026-10-21)

Working rulings, amended: the change reaches the model, and the credential is the org's (2026-10-22)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Harness: run and apply for the review Intent, and the model holds no credential that can write #194

Description

Activity

don-petry commented on Oct 21, 2026

Working rulings (overturnable in review)

don-petry commented on Oct 21, 2026

Working rulings: the Maintainer's answers (2026-10-21)

don-petry commented on Oct 21, 2026

Ruling 4, the CLI's call: --tools "" with --json-schema, settled (2026-10-21)

don-petry commented on Oct 21, 2026

Working rulings, amended: the run job's environment (2026-10-21)

don-petry commented on Oct 21, 2026

don-petry commented on Oct 22, 2026

Working rulings, amended: the change reaches the model, and the credential is the org's (2026-10-22)

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Ruling 4, the CLI's call: `--tools ""` with `--json-schema`, settled (2026-10-21)