From e0686609f5e7c8a91fde043a781a6e005c6fb7ec Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sun, 9 Aug 2026 17:28:50 -0400 Subject: [PATCH 01/17] docs(topics): lock the docker-sandbox-substrate interview Brief MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the decisions from an interview covering two questions that turned out to be one: whether to adopt a kernel-separated local isolation substrate, and what shape an autonomous multi-run delivery pipeline should take. The substrate half resolved to a non-change here. Docker Sandboxes is an instance for the sandbox-provider seam the autonomy pack already specifies, the guided-setup path already detects capability-shaped rather than instance-shaped, and the ladder forbids naming a product instance in this repository at all — so integration costs zero files. It was probed live under a hardened invocation and both ladder assertions failed inside the boundary, which is the passing condition. The pipeline half is governed by one finding: verifier independence pays at the model level rather than the context level, and judge diversity outranks judge size. Verification topology therefore becomes a configurable column on the existing guardrail matrix, keyed on roles and relational constraints rather than capability labels, which do not survive a model release. Four questions are deferred with arbiters; the sharpest is whether the ladder should model kit-supplied allow rules, since a kit widened egress on top of a global deny-all during the probe. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 106 +++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 docs/topics/docker-sandbox-substrate/PLAN.md diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md new file mode 100644 index 0000000000..3c6f3af542 --- /dev/null +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -0,0 +1,106 @@ +# docker-sandbox-substrate + +## Brief + +### TLDR + +Docker Sandboxes (`sbx`) was evaluated as an isolation substrate and as a possible route to an +agent-agnostic autonomous pipeline. It is a **substrate instance for a seam this repository already +specifies** — not a plugin, not a new repository, and not a framework. Integrating it costs **zero +files here**, because the guided-setup path already ships and the ladder forbids naming a product +instance in this repository at all. It was probed live on Windows 11 Pro under a hardened invocation +and **both isolation assertions passed**. + +The larger goal that emerged — running the full delivery lanes as separate autonomous runs — is +governed by one evidence finding that reshapes it: **verifier independence pays at the model level, +not merely the context level**, and **judge diversity outranks judge size**. Verification topology +becomes a configurable column on the existing guardrail matrix, keyed on roles and relations rather +than on capability labels, which do not survive a model release. + +### Goal + +Decide whether and how to adopt a kernel-separated local isolation substrate, and fix the shape of the +verification topology for an autonomous multi-run delivery pipeline — both grounded in current +authoritative sources rather than recall, and both expressed so that no vendor name enters this +repository's contract surface. + +### Constraints + +- **Instance names never enter this repository.** `guardrails/isolation-ladder.md` names substrate + CLASSES as marked examples only; the instance id has exactly one home — the `substrate` field in a + consumer's security binding, outside this repo. +- **Trust is earned by demonstrated boundary behavior**, never by licence, audit history, or version + number. The contract is deliberately silent on provenance and treats it as an org choice. +- **A policy the governed agents can lower is no policy.** Verification floors live on the + agent-unwritable security binding, as the per-item caps already do. +- **Independent aggregation, never deliberation.** Consensus-by-discussion measured worse than every + single-model baseline; this is a fixed invariant of the design, not a configurable knob. +- **The runner stays trigger-gated.** Nothing here fires a T4 build trigger; an off-the-shelf substrate + scales the platform wall rather than supplying the case for building past it. +- Fresh-docs mandate applies to every contract-surface change, per `CLAUDE.md`. + +### Acceptance criteria + +1. `Docker.sbx` is tracked in the dotfiles user-scope winget list, and `provisioning` is unchanged — + satisfied by `melodic-software/dotfiles#427`; the WHP optional feature proved unnecessary on a host + with Hyper-V already enabled. +2. A probe transcript exists showing **both** ladder assertions failing inside the boundary under + `--clone` + `deny-all`, with the outer context proving each target reachable/present first, and + `outer_context_networked: true` — satisfied by `.work/docker-sandbox-substrate/probe-evidence-sbx-l3.md`. +3. The isolation probe gains a **third assertion covering the workspace mount**, and its egress + assertion tests whether **data flows** rather than whether `connect()` fails. +4. Verification policy is expressed as roles + relational constraints + machine-checkable predicates, + with **no capability label** (`frontier`, `flagship`, `daily driver`) anywhere in it. +5. Per-class verification **floors** (minimum checker count, whether cross-vendor is required) live on + the security binding; lens selection and the advisory lane live in plugin `userConfig`. +6. The visual E2E lane is wired as **advisory only**, downstream of deterministic detection, and cannot + block. +7. Merge remains human-gated, with additional routing on implementer/checker disagreement; anything + auto-proceeding requires **unanimous** checker agreement. + +### Captured assumptions + +- `sbx` evidence is **version-bound** to `v0.38.0`. The product ships roughly every two weeks with an + actively changing security surface, so the probe result is a snapshot, not a standing property. +- The probe covered one kit (`shell`), one workspace, one host. The **default (non-clone) mode was not + probed** — Docker documents it as having no workspace isolation, so it is assumed to fail. +- `local-policy`'s `filesystem:read/write allow **` is assumed to govern what the host permits `sbx` to + mount, not what the guest can reach; the guest saw only the read-only workspace. Not probed directly. +- The pipeline-architecture evidence transfers from math/QA benchmarks to software delivery. The lane + flagged this as its central limitation; the software-specific evidence is thinner and newer. +- Most multi-agent papers do not run the compute-matched baseline (a five-stage pipeline versus one run + with five times the thinking budget), so measured gains are confounded by scaffold quality. + +### Out-of-scope + +- **A new repository or framework.** `sbx` scores agent-agnostic but is welded to Docker's own microVM + and is proprietary — a provider candidate, never a framework competitor. The prior peer-frameworks + verdict (compose, do not adopt wholesale) stands. +- **Building the runner.** Its design pack is complete and its build triggers are unfired. +- **Adopting an orchestrator now.** Neither agent-native option ships a gate, which is the part the + design most needs; the pipeline stays thin code against a provider seam so the choice stays + reversible. +- **Naming `sbx`, Multipass, Hyper-V, or any product in this repository's contract surface.** + +### Deferred questions + +- **Q20 — What exact form should the probe's workspace assertion take, and how should the egress + assertion be reworded to test data flow rather than `connect()` failure?** *(arbiter: + `/planning:plan`)* A connect-only probe grades this substrate wrongly: raw TCP `connect()` succeeds + because the interception layer accepts the SYN and then drops the session. +- **Q21 — Should the isolation ladder model kit-supplied allow rules?** *(arbiter: USER-RESERVED)* The + `shell` kit added `network allow openrouter.ai` on top of a global `deny-all`. Installing a kit can + widen egress; the ladder does not currently model this, and it is a supply-chain concern that + changes what a level binding certifies. +- **Q22 — Should the three genuinely-additive "software factory" concepts be recorded as deferred + items with triggers?** *(arbiter: USER-RESERVED)* Accepted in principle (Q11); the trigger wording is + unwritten. They are fleet-level economics as tracked output (partly covered by the telemetry and + return-accounting contracts), portfolio-scale multi-repo fan-out as a unit of work (absent — the + seams are per-run), and self-service golden paths for humans and agents (absent). +- **Q23 — What re-verification cadence applies to version-bound substrate evidence?** *(arbiter: + USER-RESERVED)* A ~2-week release cadence with three security fixes in three weeks means specific + flags, defaults, and guarantees rot; the durable findings are the architecture and the posture. + +## Plan + + From cddf26ac55db372e9a8fc0b4e74bc0ef7b3a5a5b Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sun, 9 Aug 2026 19:39:00 -0400 Subject: [PATCH 02/17] docs(topics): draft the docker-sandbox-substrate plan for criteria 3-7 Resolves Q20 (arbiter /planning:plan): the egress assertion becomes a certificate-verified TLS fetch, which converts the two observed false negatives into true negatives while keeping the non-zero-exit invariant; workspace containment becomes a third assertion proven from the outer side, so one rule covers both read-only and clone-mode mounts. Records the Tier A design gate as resolved by the interview register rather than re-running /planning:design, and corrects the Brief TLDR's assertion polarity to match the contract (both assertions must FAIL inside the boundary). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 310 +++++++++++++++++- .../design/design-resolution.md | 42 +++ 2 files changed, 350 insertions(+), 2 deletions(-) create mode 100644 docs/topics/docker-sandbox-substrate/design/design-resolution.md diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index 3c6f3af542..6e7641c55d 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -9,7 +9,7 @@ agent-agnostic autonomous pipeline. It is a **substrate instance for a seam this specifies** — not a plugin, not a new repository, and not a framework. Integrating it costs **zero files here**, because the guided-setup path already ships and the ladder forbids naming a product instance in this repository at all. It was probed live on Windows 11 Pro under a hardened invocation -and **both isolation assertions passed**. +and **both isolation assertions failed inside the boundary**, which is the passing condition. The larger goal that emerged — running the full delivery lanes as separate autonomous runs — is governed by one evidence finding that reshapes it: **verifier independence pays at the model level, @@ -103,4 +103,310 @@ repository's contract surface. ## Plan - +### Goal + +**What**: harden the isolation probe so it measures data flow and the workspace mount, then express +verification topology as a configurable column on the existing guardrail matrix — floors on the +agent-unwritable security binding, lens selection and the advisory visual lane in plugin +`userConfig`, merge human-gated with disagreement routing. + +**Why**: the probe currently certifies `L2` while the whole host-execution attack class goes +unmeasured, and a connect-only egress test grades a microVM-class substrate wrongly. The pipeline +half of the Brief is decided but unbuilt: without a topology column, verifier count, model routing, +and lens diversity have nowhere to bind. + +### Standards grounding + +No standards index is present in this repository, so the ladder's absent-index inference applies. +Grounded on the ambient repository standards; nothing was fetched beyond them. + +| Surface | Sections cited | Layer provenance | +|---|---|---| +| Plugin contract surface | `CLAUDE.md` — Fresh-docs mandate; Design rules (repo-agnostic, configurable without editing the plugin, plugin-form-safe, versioned) | team | +| Branch / PR process | `CLAUDE.md` — Branching & PRs (`pr-issue-linkage` body contract); `AGENTS.md` — stage explicit paths | team | +| Contract-slice lifecycle | `docs/conventions/topic-docs/README.md` — slice pruned before merge (`contract-slice-prune-gate`) | team | +| Plugin acceptance | `docs/MIGRATION-PLAYBOOK.md`, `docs/PLUGIN-PHILOSOPHY.md` | team | + +Offer, not applied: persist a standards index so future plans resolve these surfaces deterministically +instead of inferring them. + +### Q20 — RESOLVED (this plan is its arbiter) + +Both halves resolve as a **rewording plus one additive assertion**, not a loosening of any existing +check. + +**Egress — test data flow, not `connect()`.** The probe becomes a *certificate-verified* TLS fetch of +``. Certificate verification is what converts the two observed false +negatives into true negatives: an interception layer that accepts the SYN and drops the session +cannot complete the handshake, and a policy block page cannot present a certificate valid for the +probed host. The existing non-zero-exit invariant therefore stands unchanged — no relaxation — and +the transcript additionally records HOW denial occurred, in the per-target comma-separated form +`credentials_absent` already uses: + +- `egress_denied.transport_outcome` ∈ `dns-unresolved` | `connect-failed` | `tls-failed` | + `policy-intercepted`, one entry per probed host, positionally paired with `host`. +- A completed, host-authenticated TLS session that read origin application data is the FAIL + condition; it has no passing token, so it cannot be recorded as a pass. + +**Workspace — containment proven from the OUTER side.** A third assertion, `workspace_contained`, +asserting that **no write performed inside the boundary becomes visible at the host workspace path**. +Verifying from outside is what makes one rule cover both substrate shapes: a read-only mount fails +the inner write, a clone-mode mount accepts it invisibly, and both are contained. The inner exit code +is therefore deliberately NOT constrained — only host state is. + +Legs, in order: + +1. Outer, pre: the marker path is absent at the host workspace root; record the exit as + `host_pre_absent: "0"`. Record `git_config_digest_pre` (the literal `absent` where the workspace + has no `.git/config`). +2. Inner: write the marker into the workspace mount, and append to `.git/config`. Record + `inner_exit_code` for the record only. +3. Outer, post: the marker is still absent (`host_post_absent: "0"`) and + `git_config_digest_post` equals `git_config_digest_pre`. + +`.git/config` is named explicitly because it is a command key ring — `core.fsmonitor` executes host +code on `git status` — so a workspace assertion that only checked ordinary files would miss the +documented escalation path. + +### Approach + +Phase order is dependency-driven: the vocabulary leaf (Phase 2) must exist before any surface can +cite it, and Phase 1 is independent of every other phase. + +#### Phase 1: Probe hardening — data-flow egress + workspace containment [TODO] + +Review: security + +Criterion 3. Delivers the Q20 resolution above. + +- [ ] **Pre-flight consumer check** (FIRST work item — this migrates the probe-transcript contract): + `Grep` for `egress_denied`, `credentials_absent`, `outer_context_networked`, and + `verifyProbeTranscript` across `plugins/`, `scripts/`, and `.github/`; document every parse + path. Known consumers at plan time: `scripts/check-security-binding.mjs`, + `scripts/check-security-binding.fixtures.test.mjs`, the fixtures manifest, and + `skills/setup/SKILL.md`'s guardrail slice. +- [ ] `templates/isolation-probe.md` — reword the egress assertion to the certificate-verified TLS + form; add the `workspace_contained` assertion, its outer-first probe shape, and its per-substrate + wrapping note; extend the transcript capture shape with both new blocks. +- [ ] `reference/guardrails/isolation-ladder.md` — the `L2` level text names default-deny egress and + credential protection; add workspace containment as the third property a boundary must + demonstrate. Classes only, no instance names. +- [ ] `scripts/check-security-binding.mjs` — extend `verifyProbeTranscript`: validate + `egress_denied.transport_outcome` (closed token set, one entry per host, positionally paired) + and the `workspace_contained` block. **The `workspace_contained` check runs LAST in the + function** — the ~50 negative transcripts have `findings_substrings` pinned to their own + rejection reason, and an earlier-running new check would rewrite all of them. +- [ ] Transcripts referenced by PASSING fixtures gain both new blocks — exactly 6 JSON files: + `ci-pool-a-l2.json`, `ci-pool-a-l2-as112-v6.json`, `ci-pool-a-l2-multi-host.json`, + `ci-pool-a-l2-id-rsa.json`, `ci-pool-a-l3.json`, `ci-pool-a-l3-hosted.json` + (`ci-pool-a-l1-2026-07-01.txt` is an `L1` reference and is not verified). +- [ ] New NEGATIVE transcripts + manifest cases: missing `workspace_contained`; marker visible on the + host post-write; `git_config_digest` changed; missing `transport_outcome`; count mismatch + against `host`; a `data-flowed` egress outcome. +- [ ] `skills/setup/SKILL.md` — the guardrail slice's probe narration gains the third assertion. + +**Sanity Check:** + +- `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with every pre-existing case's + `findings_substrings` unchanged (diff the manifest: only ADDED keys, no MODIFIED `findings_substrings` + on pre-existing cases). +- `grep -c "workspace_contained" plugins/autonomy/skills/setup/templates/isolation-probe.md` ≥ 3. +- `grep -n "must fail to CONNECT" plugins/autonomy/skills/setup/templates/isolation-probe.md` + returns empty. +- `grep -rn "sbx\|Docker Sandboxes\|Multipass\|Hyper-V" plugins/autonomy/` returns empty. + +#### Phase 2: Verification-topology contract leaf + matrix column [TODO] + +Review: architecture + +Criterion 4, and the vocabulary Phases 3–5 cite. Documentation only — no schema, no code. + +- [ ] `reference/guardrails/verification-topology.md` (CREATE) — the normative leaf. Fixes the Q17 + vocabulary and nothing else: + **roles** `generator` · `checker` · `cross_vendor_checker` · `ranker`; + **relations** `distinct_model_from` · `distinct_vendor_from` · `not_weaker_than` (present but + NOT defaulted — no cross-vendor capability ordering exists to evaluate it against); + **predicates** `min_context_tokens` · `requires_modality` · `requires_feature`; + **budget** cost ceilings; **pin** `pinned_model_id`, append-only, reproducibility of a recorded + measurement only. + Records `frontier`, `flagship`, and `daily driver` as REJECTED with their sourced reasons, in the + same shape the ladder records its rejected trigger-source axis. + States the two fixed invariants: **independent aggregation, never deliberation**, and + **unanimous checker agreement for anything auto-proceeding**. +- [ ] `reference/guardrails.md` — add the **Verification topology** column to the matrix, its + one-line column definition, and a glance-layer routing row to the new leaf. Depth stays in the + leaf; the hub gains no prose beyond the row. + +**Sanity Check:** + +- `grep -rn "frontier\|flagship\|daily driver" plugins/autonomy/reference/` matches ONLY inside the + leaf's rejected-vocabulary section. +- `plugins/autonomy/reference/guardrails.md` matrix header row contains `Verification topology`, and + the glance-layer table contains a row pointing at `guardrails/verification-topology.md`. +- Every plugin-internal link in the new leaf resolves (`skill-reference-verify` hook passes). + +#### Phase 3: Per-class verification floors on the security binding [TODO] + +Review: security + +Criterion 5, binding half. + +- [ ] **Pre-flight consumer check** (FIRST work item): `Grep` for `verification_blocking`, + `merge_policy`, and `schema_version` parse sites across the plugin and repo scripts. +- [ ] `schemas/guardrails-security-binding.schema.json` — add `verification_topology` as an + OPTIONAL top-level key: per class, `min_checkers` (integer ≥ 1) and `cross_vendor_required` + (boolean). Optional-with-contract-defaults follows the `escalation_severity` precedent, so + `schema_version` stays `"1.0"` and all 113 existing fixtures continue to validate. Absent + binding is NOT a hole: the leaf's shipped floors apply, exactly as `escalation_severity` falls + back to contract defaults. +- [ ] `scripts/check-security-binding.mjs` — floors are FLOORS: tightening legal, weakening below the + shipped default invalid, no `override_justification` escape (the same rule + `verification_blocking` already carries). +- [ ] New fixtures: a valid tightened binding; a weakened `min_checkers`; a `cross_vendor_required` + dropped below its floor; a non-class key. + +**Sanity Check:** + +- `grep -c '"schema_version": { "const": "1.0" }' plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` + returns 1 (the version did NOT move). +- `node scripts/check-security-binding.fixtures.test.mjs` exits 0; the 22 pre-existing exit-0 cases + still exit 0 unmodified. + +#### Phase 4: Plugin `userConfig` — lens selection and the advisory visual lane [TODO] + +Review: code-design + +Criterion 5 (`userConfig` half) and criterion 6. + +- [ ] **Fresh-docs gate** (FIRST work item, non-negotiable — this is a plugin-manifest contract + change): open `docs/OFFICIAL-DOCS.md`, WebFetch + for the current `userConfig` schema and + `${user_config.KEY}` substitution rules, and cite the URL in the commit. No key is written + before that fetch. +- [ ] `.claude-plugin/plugin.json` — additive keys for lens selection and the advisory lane, each with + an explicit `default` (the plugin's existing keys all carry one). +- [ ] The visual lane is wired so it **has no blocking knob at all** — structurally incapable of + gating rather than defaulted-off-and-promotable. It is placed downstream of deterministic + detection and narrates what the deterministic layer already found. +- [ ] `reference/guardrails/verification-topology.md` — document the split: floors on the binding, + lens selection and the advisory lane in `userConfig`. Agents may raise their own verification, + never lower it. +- [ ] `skills/setup/SKILL.md` + `CHANGELOG.md` + `version` bump. + +**Sanity Check:** + +- `grep -n "advisory" plugins/autonomy/reference/guardrails/verification-topology.md` shows the visual + lane with NO `blocking` token anywhere in its section. +- `node -e "JSON.parse(require('fs').readFileSync('plugins/autonomy/.claude-plugin/plugin.json'))"` + exits 0 and every new `userConfig` entry has a `default`. +- The commit message cites the fetched docs URL. + +#### Phase 5: Merge gating and disagreement routing [TODO] + +Review: security + +Criterion 7. + +- [ ] Confirm — do not assume — that implementer/checker disagreement IS the existing + `verification-divergence` escalation event class ("a verification outcome diverges from the + expected or claimed result"). If it is, bind to it and invent no token; if it is not, the new + class binds additively as an OPTIONAL `escalation_routes` key, following the `runner-*` + precedent. +- [ ] `reference/guardrails.md` — merge-policy column keeps human merge; record that any + auto-proceeding path additionally requires unanimous checker agreement. +- [ ] `reference/guardrails/work-classes.md` — the unanimity requirement joins the promotion + discipline: a promoted `C2`/`C3` auto-merge cell still does not auto-proceed on checker dissent. +- [ ] `scripts/check-security-binding.mjs` + fixtures — a binding whose `merge_policy` is `auto` for a + class whose `verification_topology` floor cannot express unanimity is invalid. + +**Sanity Check:** + +- `grep -n "unanimous" plugins/autonomy/reference/guardrails.md plugins/autonomy/reference/guardrails/work-classes.md` + returns a match in both. +- `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with the new dissent fixtures. + +#### Phase 6: Close-out [TODO] + +- [ ] Prune `docs/topics/docker-sandbox-substrate/` in the final commit before merge + (`contract-slice-prune-gate` is a required check — a PR landing the slice on `main` can never go + green). +- [ ] Paste the approved PLAN into the PR body inside a `
` block. +- [ ] Comment the outcome on issue #2110 and close the items it lists that this work discharges. + +**Sanity Check:** `git show --stat HEAD -- docs/topics/` shows only deletions; `gh pr view --json body` +contains the PLAN block, a closing keyword, and a non-empty `## Related` section. + +### Files Affected + +| File | Action | What changes | +|---|---|---| +| `plugins/autonomy/skills/setup/templates/isolation-probe.md` | Modify | Data-flow egress rewording; third assertion; transcript shape | +| `plugins/autonomy/reference/guardrails/isolation-ladder.md` | Modify | `L2` gains workspace containment as a demonstrated property | +| `plugins/autonomy/reference/guardrails/verification-topology.md` | Create | The Q17 vocabulary leaf and its two fixed invariants | +| `plugins/autonomy/reference/guardrails.md` | Modify | Verification-topology column, definition, routing row, unanimity note | +| `plugins/autonomy/reference/guardrails/work-classes.md` | Modify | Unanimity joins the promotion discipline | +| `plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` | Modify | Optional `verification_topology` floors | +| `plugins/autonomy/skills/setup/scripts/check-security-binding.mjs` | Modify | Transcript checks; floor checks; unanimity check | +| `.../evals/fixtures/security-binding/probe-transcripts/*.json` (6) | Modify | Both new assertion blocks | +| `.../evals/fixtures/security-binding/*.json` + manifest | Create | New negative and tightened-binding cases | +| `plugins/autonomy/.claude-plugin/plugin.json` | Modify | `userConfig` keys; `version` bump | +| `plugins/autonomy/skills/setup/SKILL.md` | Modify | Probe narration; topology setup | +| `plugins/autonomy/CHANGELOG.md` | Modify | Entries per phase-PR | + +### Alternatives Considered + +| Alternative | Why rejected | +|---|---| +| Bump `schema_version` to `2.0` for the new keys | Invalidates every adopting org's binding, which fail-closes their autonomous dispatch until re-authored. The schema's own `runner-*` and `escalation_severity` keys establish optional-additive as the house pattern | +| Make the third assertion optional-when-absent | A binding could keep certifying `L2` on two-assertion evidence — the silent degrade the ladder explicitly forbids. It rides the existing UNPROVEN path instead: the binding stays valid, the level stops counting toward eligibility | +| Assert workspace containment by requiring the inner write to FAIL | Grades clone-mode substrates wrongly — they legitimately accept the write and discard it. Verifying host state from the outer side covers both shapes with one rule | +| Relax `exit_code` to allow `0` with a `policy-intercepted` outcome | Loosens a currently-strict invariant on evidence an executing agent could doctor. Certificate verification achieves the same discrimination while keeping non-zero required | +| A new escalation mechanism for checker disagreement | `verification-divergence` already exists, is already required in `escalation_routes`, and already means this. Phase 5 confirms before binding | +| Model the visual lane as a `security-review.md` layer with `blocking` defaulted off | A defaulted-off knob is promotable; measured 70% judge precision with a consistent over-crediting direction means it must never become a gate. No knob is the stronger form | + +### Test Strategy + +Test-first throughout — the fixture harness is already the red-green loop for this surface. + +- **Phase 1, 3, 5 (checker changes):** write the failing fixture FIRST (new negative transcript + + manifest case with its expected `findings_substrings`), confirm + `check-security-binding.fixtures.test.mjs` fails on it, then implement the check. This is the + established pattern for all 113 existing cases. +- **Regression floor:** the 22 pre-existing exit-0 fixtures must still exit 0, and no pre-existing + case's `findings_substrings` may change. That single assertion is what catches the check-ordering + hazard in Phase 1. +- **Phase 2 (docs only):** verification is the repo's own link and reference hooks plus the + vendor-name greps in the phase Sanity Check; no unit test applies. +- **Phase 4:** JSON parse plus a `default`-presence assertion on every new `userConfig` key; the + fresh-docs citation is verified by reading the commit message. +- **Not covered by any test:** whether the reworded egress assertion actually discriminates on a real + substrate. The existing probe evidence is the only empirical datum, and it is version-bound. Re-running + the probe against the hardened recipe is the honest verification and is called out as a risk below. + +### Risks and Mitigations + +| Risk | Likelihood | Impact | Mitigation | +|---|---|---|---| +| The new transcript check runs before existing checks and rewrites ~50 pinned `findings_substrings` | High | Med | Ordering is a stated implementation constraint; the "no pre-existing substring changed" assertion catches it mechanically | +| Certificate verification is defeated where an org installs the interceptor's CA inside the boundary | Med | High | That configuration is itself a finding; the leaf states it explicitly rather than leaving it implicit | +| Shipped floor VALUES are unevidenced | High | Med | OPEN DECISION below — not decided by this plan | +| The hardened recipe is never re-run against a real substrate, so the rewording is untested in practice | Med | Med | Re-probe is cheap (`probe-l3` still exists, stopped); recommended before Phase 1 merges | +| Pipeline evidence is pre-consensus and several findings are single-study | High | Med | `/planning:devils-advocate` at Step 4; the leaf records the evidence basis so a later result can demote a choice rather than silently contradicting it | +| Scope creep from Q21 into Phase 1 | Med | Med | Q21 is USER-RESERVED; no phase depends on it. If a phase starts to, that is drift and stops | + +### OPEN DECISIONS — not resolved by this plan + +1. **Shipped per-class floor values** (`min_checkers`, `cross_vendor_required`). The interview fixed + the SHAPE, never the numbers, and no evidence in the lanes sets them. RECOMMENDED starting floors: + `C1` 1/no · `C2` 1/no · `C3` 2/no · `C4` 3/yes · `C5` 3/yes — floors, so an org may only tighten. +2. **PR granularity.** RECOMMENDED: Phase 1 ships as its own PR (self-contained, security-bearing, + independently revertable); Phases 2–5 ship as a second PR carrying the whole topology change + coherently. Alternative: one PR per phase, five review round-trips. + +### Blast radius + +**HIGH.** The change touches an agent-unwritable security surface, a fail-closed checker with 113 +gated fixture cases, the isolation ladder's definition of `L2`, and a plugin manifest consumers +install. Triggers matched: security-sensitive surface; contract migration with downstream consumers; +fail-closed policy semantics. + diff --git a/docs/topics/docker-sandbox-substrate/design/design-resolution.md b/docs/topics/docker-sandbox-substrate/design/design-resolution.md new file mode 100644 index 0000000000..5208fb9710 --- /dev/null +++ b/docs/topics/docker-sandbox-substrate/design/design-resolution.md @@ -0,0 +1,42 @@ +--- +outcome: early-exit +tier: A +resolved_by: /planning:interview rounds 3–5 +--- + +# Design resolution — docker-sandbox-substrate + +## Tier + +**A — design-significant.** Criteria 3–7 add contract vocabulary (a verification-topology axis set), +additive keys on an agent-unwritable schema, new checker predicates, and a new plugin `userConfig` +surface. That is a contract change across four components. + +## Why `/planning:design` is not re-run + +The design threads a Tier A gate exists to force were explored and RESOLVED by `/planning:interview` +rounds 3–5, whose register gated clean (`registered=23 open=0 deferred=4 blocked=0 withdrawn=5 +answered=14 brief=ok status=clean`). Each thread below names its resolving question; the register and +its evidence live in the topic's memory slice. + +| Design thread | Resolved by | Resolution | +|---|---|---| +| Where verification policy lives (module boundary) | Q16 | SPLIT — per-class floors on the agent-unwritable security binding; lens selection and the advisory lane in plugin `userConfig` | +| Verifier-policy vocabulary (the type surface) | Q17 | Roles + relational constraints + machine-checkable predicates (Axes A–E). Capability labels rejected with sourced reasons | +| Topology shape (new mechanism vs existing seam) | Q13 refined | A column on the existing guardrail matrix, not new machinery | +| Aggregation semantics | Q19 | Unanimous to auto-proceed; any dissent routes to a human. Deliberation barred as a fixed invariant | +| Escalation shape | Q18 | Fixed per-class floor plus disagreement-triggered escalation above it | +| Visual-lane placement | Q14 | Advisory only, downstream of deterministic detection | +| Orchestrator/framework selection | Q12 | None adopted; thin pipeline code against the existing provider seam | +| Probe assertion set | Q6 | A third assertion covering the workspace mount, generalizing to every substrate class | + +## What remains open at plan time + +- **Q20** — the exact FORM of the workspace assertion and the data-flow rewording of the egress + assertion. Arbiter is `/planning:plan`; resolved in the PLAN body, not here. +- **Q21 / Q22 / Q23** — USER-RESERVED. Parked; none of criteria 3–7 depends on them. + +## Override + +If the design threads above are judged insufficiently resolved, the correction is to run +`/planning:design` before implementation — not to widen the plan. From 8f4fb87a769005724aa786237200f20b6471ecbe Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sun, 9 Aug 2026 20:15:03 -0400 Subject: [PATCH 03/17] docs(topics): revise the plan against the cross-vendor stress-test MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The same-vendor sub-agent review channel failed three times, so the independent pass ran cross-vendor instead — the route the skill names as preferred over the sub-agent fallback. Its findings were verified against the files before being applied. Material changes: the egress discriminator becomes a peer-fingerprint comparison against the outer context rather than certificate verification, which a TLS-inspection CA trusted inside the boundary would defeat; the workspace assertion is renamed workspace_host_write_contained and its post-check moves after boundary teardown; raising the L2 bar is recorded as a breaking migration that blocks dispatch for existing adopters until they re-probe; the normative leaf carries no capability label at all; the machine-checkable predicates reach the schema rather than staying prose; and criteria 6 and 7 state plainly that their runtime halves cannot be demonstrated without a runner the Brief keeps trigger-gated. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 386 +++++++++++++++---- 1 file changed, 308 insertions(+), 78 deletions(-) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index 6e7641c55d..d7e87d2c31 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -135,45 +135,76 @@ instead of inferring them. Both halves resolve as a **rewording plus one additive assertion**, not a loosening of any existing check. -**Egress — test data flow, not `connect()`.** The probe becomes a *certificate-verified* TLS fetch of -``. Certificate verification is what converts the two observed false -negatives into true negatives: an interception layer that accepts the SYN and drops the session -cannot complete the handshake, and a policy block page cannot present a certificate valid for the -probed host. The existing non-zero-exit invariant therefore stands unchanged — no relaxation — and -the transcript additionally records HOW denial occurred, in the per-target comma-separated form -`credentials_absent` already uses: - +**Egress — test data flow, not `connect()`.** The probe becomes a TLS fetch whose passing condition is +that **the inner peer is not the origin, or no origin bytes were read**. Certificate verification alone +was the first draft and is NOT sufficient: an org that installs a TLS-inspection CA inside the boundary +makes the interceptor's certificate verify cleanly, which would grade a fully-intercepted boundary as +egress-capable in one direction and a legitimately-denied one as passing in the other. The +discriminator is therefore a **peer-certificate comparison against the outer context**, which needs no +trusted-CA assumption at all: + +- The outer probe records the peer certificate fingerprint of each reachable target. +- The inner probe records its own. Denial is proven when the handshake fails, or when the inner + fingerprint DIFFERS from the outer one (an interceptor, not the origin), with zero origin bytes read. - `egress_denied.transport_outcome` ∈ `dns-unresolved` | `connect-failed` | `tls-failed` | - `policy-intercepted`, one entry per probed host, positionally paired with `host`. -- A completed, host-authenticated TLS session that read origin application data is the FAIL - condition; it has no passing token, so it cannot be recorded as a pass. - -**Workspace — containment proven from the OUTER side.** A third assertion, `workspace_contained`, -asserting that **no write performed inside the boundary becomes visible at the host workspace path**. -Verifying from outside is what makes one rule cover both substrate shapes: a read-only mount fails -the inner write, a clone-mode mount accepts it invisibly, and both are contained. The inner exit code -is therefore deliberately NOT constrained — only host state is. + `peer-substituted` | `not-applicable`, one entry per probed host, positionally paired with `host`, + in the comma-separated form `credentials_absent` already uses. +- A completed handshake whose peer fingerprint MATCHES the outer one and from which application data + was read is the FAIL condition. It has no passing token, so it cannot be recorded as a pass. + +Three additional legs close gaps a single-target exit-code test leaves open: + +- **Client readiness.** A boundary with no working TLS client would otherwise "pass" trivially — a + missing tool is indistinguishable from a denied network. The same client must first succeed against + an in-boundary endpoint; a client that cannot be shown to run yields UNPROVEN, never a pass. +- **Multiple targets, one unguessable.** One denied destination does not establish default-deny: a + policy can allow specific hosts while denying the probed one. At least two targets are required, one + of them a randomized name no allowlist could have anticipated. +- **Address family recorded, never inferred.** IPv4 and IPv6 are probed and recorded separately; an + absent family is `not-applicable`, never counted as denied. + +**Workspace — host-write containment, proven from the OUTER side.** A third assertion named +`workspace_host_write_contained` — the longer name is deliberate, because the short form would +overclaim. It asserts that **no write performed inside the boundary becomes visible at the host +workspace path**, which is Q6's write-back scope, and it does NOT cover read exposure (see the named +limitation below). Verifying from outside is what makes one rule cover both substrate shapes: a +read-only mount fails the inner write, a clone-mode mount accepts it invisibly, and both are +contained. The inner exit code is deliberately NOT constrained — only host state is. Legs, in order: -1. Outer, pre: the marker path is absent at the host workspace root; record the exit as - `host_pre_absent: "0"`. Record `git_config_digest_pre` (the literal `absent` where the workspace - has no `.git/config`). -2. Inner: write the marker into the workspace mount, and append to `.git/config`. Record - `inner_exit_code` for the record only. -3. Outer, post: the marker is still absent (`host_post_absent: "0"`) and - `git_config_digest_post` equals `git_config_digest_pre`. +1. Outer, pre: snapshot the host workspace. Record `git_config_digest_pre` (the literal `absent` + where the workspace has no `.git/config`) and confirm every canary path is absent. +2. Inner: write a **randomized canary** at each of a small fixed set of shapes — a plain file, a + dotfile, and a `.git/`-relative path — and append to `.git/config`. Randomized names defeat a + substrate that special-cases a known probe path; the shape set covers case-folding and + hidden-file handling that a single literal path would miss. Record `inner_exit_code` for the + record only. +3. **Tear the boundary down, then** re-check. An immediate post-check can miss delayed propagation + on a caching or asynchronously-flushed mount, so the post-check runs after teardown, with paths + canonicalized on both sides before comparison. +4. Outer, post: every canary still absent (`host_post_absent: "0"`) and `git_config_digest_post` + equals `git_config_digest_pre`. `.git/config` is named explicitly because it is a command key ring — `core.fsmonitor` executes host -code on `git status` — so a workspace assertion that only checked ordinary files would miss the -documented escalation path. +code on `git status` — so an assertion that only checked ordinary files would miss the documented +escalation path. + +**Where the host workspace path is not observable from the outer context** (a hosted ephemeral +executor whose storage the operator cannot inspect), the assertion records `not-applicable` and the +level is UNPROVEN for that surface. It never silently passes. + +**Named limitation, carried into the leaf, not hidden.** This assertion does not measure READ +exposure, and clone mode leaves reads fully open — so exfiltration of workspace contents is +unaffected by a passing result. Q6 scoped the assertion to write-back; widening it to reads is +recorded as a deferred item with its own trigger rather than implied by the assertion's name. ### Approach Phase order is dependency-driven: the vocabulary leaf (Phase 2) must exist before any surface can cite it, and Phase 1 is independent of every other phase. -#### Phase 1: Probe hardening — data-flow egress + workspace containment [TODO] +#### Phase 1: Probe hardening — data-flow egress + workspace host-write containment [TODO] Review: security @@ -185,35 +216,60 @@ Criterion 3. Delivers the Q20 resolution above. path. Known consumers at plan time: `scripts/check-security-binding.mjs`, `scripts/check-security-binding.fixtures.test.mjs`, the fixtures manifest, and `skills/setup/SKILL.md`'s guardrail slice. -- [ ] `templates/isolation-probe.md` — reword the egress assertion to the certificate-verified TLS - form; add the `workspace_contained` assertion, its outer-first probe shape, and its per-substrate - wrapping note; extend the transcript capture shape with both new blocks. +- [ ] **Hardcoded assertion COUNTS must move with the assertion set** — verified present at + `templates/isolation-probe.md:6` ("the SAME two assertions"), `:13` ("Two checks"), `:103` + ("both assertions"), and `schemas/guardrails-security-binding.schema.json:178` ("capture shape + with both assertions"). Leaving any of them stale makes the contract self-contradicting. +- [ ] `templates/isolation-probe.md` — reword the egress assertion to the peer-comparison form and add + its readiness, multi-target, and address-family legs; add the + `workspace_host_write_contained` assertion with its outer-first, post-teardown probe shape and + per-substrate wrapping note; extend the transcript capture shape with both new blocks. - [ ] `reference/guardrails/isolation-ladder.md` — the `L2` level text names default-deny egress and - credential protection; add workspace containment as the third property a boundary must - demonstrate. Classes only, no instance names. + credential protection; add host-write containment as the third property a boundary must + demonstrate, and state the read-exposure limitation in the same breath. Classes only, no + instance names. - [ ] `scripts/check-security-binding.mjs` — extend `verifyProbeTranscript`: validate `egress_denied.transport_outcome` (closed token set, one entry per host, positionally paired) - and the `workspace_contained` block. **The `workspace_contained` check runs LAST in the - function** — the ~50 negative transcripts have `findings_substrings` pinned to their own - rejection reason, and an earlier-running new check would rewrite all of them. + and the `workspace_host_write_contained` block. **Both new checks run LAST in the function** — + `verifyProbeTranscript` returns the FIRST problem it finds, and all 58 `probe-evidence-*` + fixture cases pin a `findings_substrings` naming their own specific rejection reason (verified: + `records assertions.egress_denied.host "192.88.99.1"` and the like). A new check running + earlier would return the new reason instead and rewrite every one of them. - [ ] Transcripts referenced by PASSING fixtures gain both new blocks — exactly 6 JSON files: `ci-pool-a-l2.json`, `ci-pool-a-l2-as112-v6.json`, `ci-pool-a-l2-multi-host.json`, `ci-pool-a-l2-id-rsa.json`, `ci-pool-a-l3.json`, `ci-pool-a-l3-hosted.json` (`ci-pool-a-l1-2026-07-01.txt` is an `L1` reference and is not verified). -- [ ] New NEGATIVE transcripts + manifest cases: missing `workspace_contained`; marker visible on the - host post-write; `git_config_digest` changed; missing `transport_outcome`; count mismatch - against `host`; a `data-flowed` egress outcome. +- [ ] New NEGATIVE transcripts + manifest cases: missing `workspace_host_write_contained`; a canary + visible on the host post-teardown; `git_config_digest` changed; missing `transport_outcome`; + count mismatch against `host`; a matching peer fingerprint with data read; a single-target + egress probe; an unproven TLS client. - [ ] `skills/setup/SKILL.md` — the guardrail slice's probe narration gains the third assertion. +- [ ] **Staged activation — this raises the `L2` bar for surfaces already bound.** Every deployed `L2` + binding was probed under the two-assertion recipe, so its transcript has no third block; the new + check makes those levels UNPROVEN, and the ladder's fail-closed rule then BLOCKS autonomous + dispatch on that surface. `L3` inherits it. That is the correct security outcome and it is a + breaking migration for every existing adopter, so it ships staged, never silently: + (a) the release notes and `CHANGELOG` state the bar raise and what re-probing costs; + (b) the checker's UNPROVEN finding text names the third assertion as the cause and points at the + updated recipe, so an operator reads a remedy rather than a bare rejection; + (c) the plugin `version` bump is MINOR at minimum, and the CHANGELOG entry is written as a + migration note. In-repo fixtures are NOT the migration — they are only the test of it. +- [ ] **Merge gate — re-probe on a real substrate.** The reworded egress assertion has never been run; + the existing evidence was captured under the OLD recipe. The stopped `probe-l3` sandbox is still + on this machine, so re-probing is cheap. At least one real substrate must be re-probed under the + hardened recipe before this phase merges, and the transcript committed as updated evidence. **Sanity Check:** - `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with every pre-existing case's `findings_substrings` unchanged (diff the manifest: only ADDED keys, no MODIFIED `findings_substrings` on pre-existing cases). -- `grep -c "workspace_contained" plugins/autonomy/skills/setup/templates/isolation-probe.md` ≥ 3. -- `grep -n "must fail to CONNECT" plugins/autonomy/skills/setup/templates/isolation-probe.md` +- `grep -c "workspace_host_write_contained" plugins/autonomy/skills/setup/templates/isolation-probe.md` ≥ 3. +- `grep -rn "must fail to CONNECT\|SAME two assertions\|Two checks\|both assertions" plugins/autonomy/skills/setup/` returns empty. - `grep -rn "sbx\|Docker Sandboxes\|Multipass\|Hyper-V" plugins/autonomy/` returns empty. +- A re-probe transcript exists under `.work/docker-sandbox-substrate/` recording the new + `transport_outcome` and `workspace_host_write_contained` blocks from a live run. #### Phase 2: Verification-topology contract leaf + matrix column [TODO] @@ -229,8 +285,12 @@ Criterion 4, and the vocabulary Phases 3–5 cite. Documentation only — no sch **predicates** `min_context_tokens` · `requires_modality` · `requires_feature`; **budget** cost ceilings; **pin** `pinned_model_id`, append-only, reproducibility of a recorded measurement only. - Records `frontier`, `flagship`, and `daily driver` as REJECTED with their sourced reasons, in the - same shape the ladder records its rejected trigger-source axis. + **The leaf contains no capability label at all** — not even as a rejected example. Criterion 4 + says "no capability label anywhere in it", and a rejected-vocabulary section would still put the + words in the normative artifact. The leaf states only that capability labels are rejected as + policy vocabulary, because they name a different thing at each vendor and do not survive a model + release; the sourced per-label rationale lives in the PR body and issue #2110, which is where a + future reader tempted to reintroduce one will find it. States the two fixed invariants: **independent aggregation, never deliberation**, and **unanimous checker agreement for anything auto-proceeding**. - [ ] `reference/guardrails.md` — add the **Verification topology** column to the matrix, its @@ -239,8 +299,8 @@ Criterion 4, and the vocabulary Phases 3–5 cite. Documentation only — no sch **Sanity Check:** -- `grep -rn "frontier\|flagship\|daily driver" plugins/autonomy/reference/` matches ONLY inside the - leaf's rejected-vocabulary section. +- `grep -rniE "frontier|flagship|daily driver" plugins/autonomy/reference/` returns EMPTY — criterion + 4's "anywhere in it" is a zero-match assertion, not a scoped one. - `plugins/autonomy/reference/guardrails.md` matrix header row contains `Verification topology`, and the glance-layer table contains a row pointing at `guardrails/verification-topology.md`. - Every plugin-internal link in the new leaf resolves (`skill-reference-verify` hook passes). @@ -254,16 +314,23 @@ Criterion 5, binding half. - [ ] **Pre-flight consumer check** (FIRST work item): `Grep` for `verification_blocking`, `merge_policy`, and `schema_version` parse sites across the plugin and repo scripts. - [ ] `schemas/guardrails-security-binding.schema.json` — add `verification_topology` as an - OPTIONAL top-level key: per class, `min_checkers` (integer ≥ 1) and `cross_vendor_required` - (boolean). Optional-with-contract-defaults follows the `escalation_severity` precedent, so - `schema_version` stays `"1.0"` and all 113 existing fixtures continue to validate. Absent - binding is NOT a hole: the leaf's shipped floors apply, exactly as `escalation_severity` falls - back to contract defaults. + OPTIONAL top-level key. **Criterion 4 says the policy is expressed as roles + relations + + MACHINE-CHECKABLE predicates; a predicate that never reaches the schema is not machine-checkable, + so all three axes are modeled here, not only the count.** Per class: `min_checkers` (integer ≥ 1), + `cross_vendor_required` (boolean), and a role list whose entries carry the relational constraints + (`distinct_model_from`, `distinct_vendor_from`) and the predicates (`min_context_tokens`, + `requires_modality`, `requires_feature`). Optional-with-contract-defaults follows the + `escalation_severity` precedent, so `schema_version` stays `"1.0"` and all 113 existing fixtures + continue to validate. Absent binding is NOT a hole: the leaf's shipped floors apply, exactly as + `escalation_severity` falls back to contract defaults. - [ ] `scripts/check-security-binding.mjs` — floors are FLOORS: tightening legal, weakening below the shipped default invalid, no `override_justification` escape (the same rule - `verification_blocking` already carries). + `verification_blocking` already carries). A relational constraint naming a role absent from its + own class's role list is invalid — that check is what makes the predicate machine-checkable + rather than decorative. - [ ] New fixtures: a valid tightened binding; a weakened `min_checkers`; a `cross_vendor_required` - dropped below its floor; a non-class key. + dropped below its floor; a non-class key; a `distinct_vendor_from` pointing at an undeclared + role; an unknown predicate token. **Sanity Check:** @@ -271,6 +338,11 @@ Criterion 5, binding half. returns 1 (the version did NOT move). - `node scripts/check-security-binding.fixtures.test.mjs` exits 0; the 22 pre-existing exit-0 cases still exit 0 unmodified. +- Every axis named in the leaf appears as a schema key: for each of `min_checkers`, + `cross_vendor_required`, `distinct_model_from`, `distinct_vendor_from`, `min_context_tokens`, + `requires_modality`, `requires_feature`, `grep -c` in the schema returns ≥ 1. +- The undeclared-role fixture exits 1 with its pinned finding — proof the relational constraint is + enforced, not merely declared. #### Phase 4: Plugin `userConfig` — lens selection and the advisory visual lane [TODO] @@ -293,10 +365,19 @@ Criterion 5 (`userConfig` half) and criterion 6. never lower it. - [ ] `skills/setup/SKILL.md` + `CHANGELOG.md` + `version` bump. +**Honest limit.** Criterion 6 says the lane "cannot block". With no runner built, there is no runtime +in which to demonstrate that, and building one is barred by the Brief's own trigger-gate constraint. +So the criterion is met by **structural impossibility rather than a runtime test**: the lane is given +no blocking knob in `userConfig` and no `VerificationKnob` cell in the schema, so there is nothing an +org could flip. The runtime ordering assertion — deterministic pass plus visual fail still advances — +is recorded as a deferred item bound to the runner's build trigger, not claimed here. + **Sanity Check:** -- `grep -n "advisory" plugins/autonomy/reference/guardrails/verification-topology.md` shows the visual - lane with NO `blocking` token anywhere in its section. +- The visual lane's section in `reference/guardrails/verification-topology.md` contains no + `blocking` token: `grep -c blocking` over that section returns 0. +- `grep -c "visual" plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` + returns 0 — the lane has no binding-side knob at all, which is what makes it unpromotable. - `node -e "JSON.parse(require('fs').readFileSync('plugins/autonomy/.claude-plugin/plugin.json'))"` exits 0 and every new `userConfig` entry has a `default`. - The commit message cites the fetched docs URL. @@ -307,22 +388,39 @@ Review: security Criterion 7. -- [ ] Confirm — do not assume — that implementer/checker disagreement IS the existing - `verification-divergence` escalation event class ("a verification outcome diverges from the - expected or claimed result"). If it is, bind to it and invent no token; if it is not, the new - class binds additively as an OPTIONAL `escalation_routes` key, following the `runner-*` - precedent. -- [ ] `reference/guardrails.md` — merge-policy column keeps human merge; record that any - auto-proceeding path additionally requires unanimous checker agreement. +**Routing question — RESOLVED, not deferred.** Verified this session: `verification-divergence` is +already a REQUIRED key in `escalation_routes`, and `guardrails.md` defines it as "a verification +outcome diverges from the expected or claimed result" — which is exactly implementer/checker +disagreement. Phase 5 binds to it and invents no token. The new machinery is the **unanimity +predicate**, not the event class. + +- [ ] `reference/guardrails.md` — merge-policy column keeps human merge; record that **every automatic + transition**, not merely a merge, requires unanimous checker agreement. Scoping it to merge alone + would leave an auto-advancing pipeline stage ungoverned, which is the hole criterion 7 exists to + close. - [ ] `reference/guardrails/work-classes.md` — the unanimity requirement joins the promotion discipline: a promoted `C2`/`C3` auto-merge cell still does not auto-proceed on checker dissent. + Promotion does not survive dissent; it is a ceiling, and dissent lowers the effective state the + same way contrary evidence already does. - [ ] `scripts/check-security-binding.mjs` + fixtures — a binding whose `merge_policy` is `auto` for a class whose `verification_topology` floor cannot express unanimity is invalid. +**Honest limit.** Criterion 7's "unanimous agreement to auto-proceed" is a RUNTIME aggregation rule, +and the runner that would aggregate is design-only and trigger-gated — `runner.md` states "no build +begins until a T4 build trigger fires". This phase therefore delivers what is checkable without a +runtime: the contract obligation, the promotion-discipline consequence, and a binding-validity check +that a class cannot be configured to auto-proceed without a floor capable of expressing unanimity. +The per-run verdict-aggregation gate (unanimous pass, single dissent, checker timeout, duplicate +checker identity) is specified as a runner-seam obligation and lands with the runner build. The plan +does not claim enforcement it cannot demonstrate. + **Sanity Check:** - `grep -n "unanimous" plugins/autonomy/reference/guardrails.md plugins/autonomy/reference/guardrails/work-classes.md` - returns a match in both. + returns a match in both — a documentation assertion, and labeled as such. +- The BEHAVIOR assertion: a new fixture binding `merge_policy.C3 = "auto"` with a + `verification_topology` floor that cannot express unanimity exits 1 with its pinned finding, and the + otherwise-identical binding with a conforming floor exits 0. - `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with the new dissent fixtures. #### Phase 6: Close-out [TODO] @@ -360,8 +458,10 @@ contains the PLAN block, a closing keyword, and a non-empty `## Related` section | Bump `schema_version` to `2.0` for the new keys | Invalidates every adopting org's binding, which fail-closes their autonomous dispatch until re-authored. The schema's own `runner-*` and `escalation_severity` keys establish optional-additive as the house pattern | | Make the third assertion optional-when-absent | A binding could keep certifying `L2` on two-assertion evidence — the silent degrade the ladder explicitly forbids. It rides the existing UNPROVEN path instead: the binding stays valid, the level stops counting toward eligibility | | Assert workspace containment by requiring the inner write to FAIL | Grades clone-mode substrates wrongly — they legitimately accept the write and discard it. Verifying host state from the outer side covers both shapes with one rule | -| Relax `exit_code` to allow `0` with a `policy-intercepted` outcome | Loosens a currently-strict invariant on evidence an executing agent could doctor. Certificate verification achieves the same discrimination while keeping non-zero required | -| A new escalation mechanism for checker disagreement | `verification-divergence` already exists, is already required in `escalation_routes`, and already means this. Phase 5 confirms before binding | +| Certificate verification alone as the egress discriminator (this plan's own first draft) | An org that installs a TLS-inspection CA inside the boundary makes the interceptor verify cleanly, so the check grades an intercepted boundary as egress-capable. Replaced by a peer-fingerprint comparison against the outer context, which assumes no trusted CA | +| An origin-signed nonce challenge verified against an embedded public key | Strictly stronger, but no well-known public endpoint will sign a caller-supplied nonce, so it cannot be substrate-agnostic or vendor-neutral — it would require shipping and operating an endpoint, which this repository has no business doing | +| A new escalation mechanism for checker disagreement | `verification-divergence` already exists, is already required in `escalation_routes`, and its definition already means this — verified this session, so Phase 5 binds rather than deferring | +| Prohibiting automatic merge outright to satisfy criterion 7 | Overshoots the criterion, which permits auto-proceeding on unanimity, and would revoke the shipped `C2`/`C3` promotion path the guardrail matrix already grants. Unanimity is scoped to every automatic transition instead | | Model the visual lane as a `security-review.md` layer with `blocking` defaulted off | A defaulted-off knob is promotable; measured 70% judge precision with a consistent over-crediting direction means it must never become a gate. No knob is the stronger form | ### Test Strategy @@ -379,34 +479,164 @@ Test-first throughout — the fixture harness is already the red-green loop for vendor-name greps in the phase Sanity Check; no unit test applies. - **Phase 4:** JSON parse plus a `default`-presence assertion on every new `userConfig` key; the fresh-docs citation is verified by reading the commit message. -- **Not covered by any test:** whether the reworded egress assertion actually discriminates on a real - substrate. The existing probe evidence is the only empirical datum, and it is version-bound. Re-running - the probe against the hardened recipe is the honest verification and is called out as a risk below. +- **Phase 5:** the documentation greps are labeled as documentation assertions; the behavior assertion + is the paired fixture (auto-merge with a unanimity-incapable floor exits 1; the conforming twin + exits 0). +- **Covered only by a live re-probe, not by the fixture harness:** whether the reworded egress and + workspace assertions actually discriminate on a real substrate. Fixtures test the CHECKER, never the + RECIPE. This is why the live re-probe is a Phase 1 merge gate rather than a suggestion. +- **Not covered at all, and stated rather than hidden:** runtime behavior for criteria 6 and 7. No + runner exists to exercise them, and building one is barred by the Brief's trigger-gate constraint. ### Risks and Mitigations | Risk | Likelihood | Impact | Mitigation | |---|---|---|---| -| The new transcript check runs before existing checks and rewrites ~50 pinned `findings_substrings` | High | Med | Ordering is a stated implementation constraint; the "no pre-existing substring changed" assertion catches it mechanically | -| Certificate verification is defeated where an org installs the interceptor's CA inside the boundary | Med | High | That configuration is itself a finding; the leaf states it explicitly rather than leaving it implicit | -| Shipped floor VALUES are unevidenced | High | Med | OPEN DECISION below — not decided by this plan | -| The hardened recipe is never re-run against a real substrate, so the rewording is untested in practice | Med | Med | Re-probe is cheap (`probe-l3` still exists, stopped); recommended before Phase 1 merges | -| Pipeline evidence is pre-consensus and several findings are single-study | High | Med | `/planning:devils-advocate` at Step 4; the leaf records the evidence basis so a later result can demote a choice rather than silently contradicting it | +| **Phase 1 blocks autonomous dispatch for every existing adopter.** Their `L2` transcripts predate the third assertion, so the levels go UNPROVEN and the ladder's fail-closed rule blocks the surface. `L3` inherits it | **Certain** | **High** | This is the intended security outcome of a bar raise, but it is breaking. Staged activation is a Phase 1 work item: remedy-bearing finding text, a migration CHANGELOG note, and a MINOR-at-minimum version bump. In-repo fixtures are the test of the migration, never the migration itself | +| The new transcript checks run before existing checks and rewrite 58 pinned `findings_substrings` | High | Med | Last-position ordering is a stated implementation constraint; the "no pre-existing substring changed" assertion catches it mechanically | +| A TLS-inspection CA trusted inside the boundary makes an interceptor's certificate verify cleanly | Med | High | Why the assertion is a peer-fingerprint COMPARISON against the outer context rather than plain certificate verification — it needs no trusted-CA assumption. First-draft cert-verification-only was rejected for exactly this | +| A boundary with no working TLS client "passes" trivially — a missing tool looks like a denied network | Med | High | The client-readiness leg: the same client must succeed against an in-boundary endpoint first, or the level is UNPROVEN | +| One denied target certifies default-deny while policy quietly allows others | Med | High | Two targets minimum, one a randomized name no allowlist anticipated. Note the adjacent kit-widening question is Q21 and stays USER-RESERVED — this leg strengthens the probe without deciding it | +| **The assertion does not cover READ exposure, and clone mode leaves reads fully open** | Certain | Med | Named limitation carried in the assertion's own name (`workspace_host_write_contained`) and stated in the leaf. Widening to reads is a deferred item with a trigger, not an implied guarantee | +| A caching or async mount propagates the inner write after the post-check | Med | High | Post-check runs after boundary teardown, with both sides canonicalized | +| Shipped floor VALUES are unevidenced | High | Med | OPEN DECISION 1 below — BLOCKS Phase 3, not the plan's approval | +| The hardened recipe is never re-run against a real substrate | Med | High | Promoted from advisory to a Phase 1 MERGE GATE — the stopped `probe-l3` sandbox is still on this machine | +| Criteria 6 and 7 are runtime claims with no runtime to test them in | Certain | Med | Both are met by structural impossibility plus contract obligation, and each phase states the limit explicitly. The runtime assertions are recorded as runner-seam obligations bound to the build trigger — not claimed as delivered | +| Pipeline evidence is pre-consensus and several findings are single-study | High | Med | The leaf records each choice's evidence basis, so a later contrary result demotes that choice explicitly rather than silently contradicting a rule with no stated warrant | | Scope creep from Q21 into Phase 1 | Med | Med | Q21 is USER-RESERVED; no phase depends on it. If a phase starts to, that is drift and stops | ### OPEN DECISIONS — not resolved by this plan -1. **Shipped per-class floor values** (`min_checkers`, `cross_vendor_required`). The interview fixed - the SHAPE, never the numbers, and no evidence in the lanes sets them. RECOMMENDED starting floors: - `C1` 1/no · `C2` 1/no · `C3` 2/no · `C4` 3/yes · `C5` 3/yes — floors, so an org may only tighten. -2. **PR granularity.** RECOMMENDED: Phase 1 ships as its own PR (self-contained, security-bearing, - independently revertable); Phases 2–5 ship as a second PR carrying the whole topology change - coherently. Alternative: one PR per phase, five review round-trips. +1. **Shipped per-class floor values** (`min_checkers`, `cross_vendor_required`). **BLOCKS Phase 3**, + not this plan's approval. The interview fixed the SHAPE, never the numbers, and no research lane + sets them — so choosing them here would be a sizing guess dressed as evidence. RECOMMENDED starting + floors: `C1` 1/no · `C2` 1/no · `C3` 2/no · `C4` 3/yes · `C5` 3/yes. They are FLOORS, so an org may + only tighten, and the cross-vendor requirement lands on exactly the two classes whose cost the + evidence justifies. +2. **PR granularity.** RECOMMENDED: Phase 1 ships as its own PR — it is self-contained, + security-bearing, independently revertable, and it carries a breaking migration that deserves its + own release note. Phases 2–5 ship as a second PR carrying the topology change coherently. + Alternative: one PR per phase, five review round-trips. + +### Deferred, with triggers — recorded so they are not silently implied + +- **Workspace READ-exposure assertion.** Trigger: any adopter binds a substrate whose workspace mount + is readable and whose threat model includes workspace exfiltration. Q6 scoped this round to + write-back; the assertion's name says so. +- **Runtime verdict-aggregation gate** (unanimous pass, single dissent, checker timeout, duplicate + checker identity). Trigger: the runner's T4 build trigger fires. Specified as a runner-seam + obligation in Phase 5; not deliverable before a runtime exists. +- **Runtime advisory-ordering assertion** (deterministic pass plus visual fail still advances). Same + trigger. + +These are distinct from Q21/Q22/Q23, which are USER-RESERVED and belong to the human, not to a +trigger. ### Blast radius **HIGH.** The change touches an agent-unwritable security surface, a fail-closed checker with 113 gated fixture cases, the isolation ladder's definition of `L2`, and a plugin manifest consumers install. Triggers matched: security-sensitive surface; contract migration with downstream consumers; -fail-closed policy semantics. +fail-closed policy semantics. Phase 1 additionally halts autonomous dispatch for every existing +adopter until they re-probe, which is a breaking migration on a security floor. + +### Stress-test summary + +Two independent passes were attempted; one channel worked. + +- **Fresh-context sub-agent review (Step 3): FAILED TO DELIVER.** Three separate spawns each returned + an idle notification with no report. The subagent return channel is broken in this session. Recorded + rather than papered over, because Step 3 is mandatory and a silent skip would be the failure mode + the step exists to prevent. +- **Cross-vendor review (Codex): DELIVERED.** This is the route the skill names as PREFERRED over the + same-vendor sub-agent, so the fallback failing did not cost independence. Its findings drove the + revisions above: the TLS-inspection defeat of certificate verification, the client-readiness and + multi-target gaps, the async-propagation and canary-shape gaps in the workspace assertion, the + unaccounted `L2` migration, the documentation-only predicates, the capability labels leaking into + the normative leaf, and Phase 5 testing expressibility rather than behavior. +- **Findings verified before applying, not taken on trust.** Confirmed against the files: the + hardcoded assertion counts at `isolation-probe.md:6,13,103` and `schema:178`; criterion 4's literal + "anywhere in it"; `runner.md`'s "no build begins" (which is what makes criteria 6–7 runtime-untestable). + Confirmed harmless: `human-gated-only-no-l2.json` binds only `L1` and has an empty + `findings_substrings`, so the bar raise does not flip it. +- **Findings REJECTED with reasons:** prohibiting automatic merge outright (overshoots criterion 7 and + revokes a shipped promotion path) and the origin-signed-nonce challenge (cannot be vendor-neutral + without operating an endpoint). Both are recorded in Alternatives Considered. + +### Execution shape + +Phase 1 is file-disjoint from Phase 2 and depends on nothing; every other phase is gated. + +| Phase | Files | Overlaps with | +|---|---|---| +| 1 | probe template, isolation-ladder, checker, manifest, fixtures, transcripts, SKILL, CHANGELOG | 3, 5 (checker, manifest, fixtures) · 4 (SKILL, CHANGELOG) | +| 2 | guardrails.md, verification-topology.md (new) | 3, 4 (leaf) · 5 (guardrails.md) | +| 3 | schema, checker, manifest, fixtures, leaf | 1, 5 · 2, 4 | +| 4 | plugin.json, SKILL, leaf, CHANGELOG | 1 · 2, 3 | +| 5 | guardrails.md, work-classes.md, checker, fixtures | 1, 3 · 2 | + +Dependency graph: Phase 2 defines the vocabulary Phases 3–5 cite, so 2 gates all three. Phase 3's +schema is what Phase 5's binding-validity check reads, so 3 gates 5. **Phase 1 is independent of every +other phase.** + +**Recommended shape: sequential, 2 → 3 → 4 → 5, with Phase 1 free to run concurrently.** Phase 1 and +Phase 2 are genuinely file-disjoint, and under the recommended PR granularity they land in separate +PRs anyway — so concurrency there is free rather than orchestrated. Within the 2–5 chain the file +overlap on the checker and the topology leaf is heavy enough that parallelism would cost more in +conflict handling than it saves. + +| Phase | Surface | Basis | +|---|---|---| +| 1 | main-session | Security-bearing contract change with a breaking migration; judgment-heavy throughout | +| 2 | main-session | Normative contract prose; the vocabulary every later phase cites | +| 3 | main-session | Schema plus checker semantics on the agent-unwritable surface | +| 4 | main-session | Gated on a live docs fetch and a manifest contract change | +| 5 | main-session | Promotion-discipline semantics; the highest-consequence cell in the matrix | +| 6 | main-session | Close-out, prune, PR body, issue comment | + +No phase routes to a sub-agent worker. Two reasons, both real: every phase is judgment-heavy contract +work rather than mechanical volume, and the sub-agent return channel demonstrably failed three times +in this session. If a later session finds the channel healthy, Phase 1's fixture authoring is the one +slice that would delegate cleanly. + +### Decisions made (gate-passed) + +| Decision | What it changes in the plan | Basis (evidence) | +|---|---|---| +| `[EXEC-SHAPE]` Peer-fingerprint comparison, not certificate verification, as the egress discriminator | The Q20 egress resolution and the `transport_outcome` token set | Certificate verification is defeated by a TLS-inspection CA trusted inside the boundary; a comparison against the outer context's fingerprint assumes no trusted CA at all | +| `[EXEC-SHAPE]` The workspace assertion is named `workspace_host_write_contained` | The assertion name, the leaf text, and the deferred read-exposure item | Q6 scoped this round to write-back, and clone mode leaves reads fully open — the short name would have implied coverage the assertion does not provide | +| `[EXEC-SHAPE]` Post-check runs after boundary teardown, with randomized canaries across three path shapes | Phase 1's probe shape | A caching or asynchronously-flushed mount can propagate after an immediate check; a single literal path misses case-folding and hidden-file handling | +| `[EXEC-SHAPE]` Optional-additive schema keys; `schema_version` stays `"1.0"` | Phase 3's schema change and the 113-fixture regression floor | `escalation_severity` and the `runner-*` keys are the house precedent, described in the schema as preserving existing bindings. Verified: `const "1.0"` | +| `[EXEC-SHAPE]` New transcript checks run LAST in `verifyProbeTranscript` | Phase 1's implementation constraint and its Sanity Check | The function returns the first problem found, and all 58 `probe-evidence-*` cases pin their own rejection reason | +| `[EXEC-SHAPE]` The normative leaf contains no capability label at all, not even a rejected one | Phase 2's leaf content and its zero-match Sanity Check | Criterion 4's literal text is "no capability label ... anywhere in it"; the sourced rationale moves to the PR body and #2110 | +| `[FALLBACK — confirm or override]` Staged activation rather than an immediate hard cutover for the `L2` bar raise | A new Phase 1 work item and the top Risks row | The Brief did not anticipate that raising `L2` blocks dispatch for existing adopters. Fail-closed is correct; shipping it without a migration note is not | +| `[FALLBACK — confirm or override]` Criteria 6 and 7 are met by structural impossibility plus contract obligation, with the runtime assertions deferred to the runner build | The honest-limit notes in Phases 4 and 5, and two deferred items | `runner.md`: "no build begins until a T4 build trigger fires", and the Brief locks that constraint. The alternative would be claiming enforcement that cannot be demonstrated | +| `[EXEC-SHAPE]` Live re-probe promoted from advisory to a Phase 1 merge gate | Phase 1's merge gate and the Test Strategy | Fixtures test the checker, never the recipe; the reworded assertions have never been run against a real boundary | +| `[EXEC-SHAPE]` The Tier A design gate is satisfied by the interview register rather than re-running `/planning:design` | `design/design-resolution.md` exists instead of a design pack | Rounds 3–5 resolved every design thread and the register gated clean; each thread is mapped to its resolving question in that file | + +### Open questions + +- OPEN DECISION 1 (floor values) must resolve before Phase 3 starts. Nothing else is blocked. +- Q21, Q22, Q23 remain USER-RESERVED. No phase depends on any of them, and Phase 1's multi-target + egress leg deliberately strengthens the probe without deciding Q21. + +### Handoff to implementation + +#### User-approval gates + +- Both `[FALLBACK]` rows above, before the phase that implements them. +- OPEN DECISION 1, before Phase 3. +- Phase 1's merge gate: the live re-probe transcript is reviewed before the phase merges, because it + is the only evidence the reworded recipe works. + +#### Execution shape + +Sequential 2 → 3 → 4 → 5, Phase 1 concurrent and independent, all main-session. No scope-fencing +tables — no phase is delegated. + +#### Mechanical work + +Commit boundaries follow phases. Stage explicit paths, never `git add -A`. The contract slice +`docs/topics/docker-sandbox-substrate/` is pruned in the final commit before merge, and the PR body +carries the closing keyword plus a non-empty `## Related` section or CI fails on the linkage check. From de01c3706eca1d8150c74ffe89710b35e518b468 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sun, 9 Aug 2026 20:16:34 -0400 Subject: [PATCH 04/17] docs(topics): scope the stress-test claim to what it actually establishes The cross-vendor pass was an independent reading, not an execution: no fixture harness run, no substrate re-probed. It inherits the same limit the Test Strategy already states, which is why the live re-probe stays a merge gate. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index d7e87d2c31..f534d7c1b4 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -562,6 +562,10 @@ Two independent passes were attempted; one channel worked. - **Findings REJECTED with reasons:** prohibiting automatic merge outright (overshoots criterion 7 and revokes a shipped promotion path) and the origin-signed-nonce challenge (cannot be vendor-neutral without operating an endpoint). Both are recorded in Alternatives Considered. +- **What the stress-test does NOT establish.** It was an independent READING of the plan against the + files. Nothing was executed: no fixture harness run, no substrate re-probed, no checker exercised. + The review inherits the Test Strategy's own limit — it can find a wrong plan, not prove a right one. + The live re-probe merge gate exists because no amount of review substitutes for running the recipe. ### Execution shape From 0b1d639db305ca4c224eb5d392f880e73548f6d1 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Mon, 10 Aug 2026 11:28:06 -0400 Subject: [PATCH 05/17] docs(topics): carry Phase 1's outcome and review lessons onto the plan branch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This branch is the durable home for phases 2-5, and it had fallen behind the artifact: it was missing both dated corrections, the scope change on randomized egress targets, the Phase 1 closure, and the live-probe result. Adds the PR review outcome, because one of its lessons is load-bearing for Phase 3. Four of the five findings shared a single shape — counting entries where coverage was meant — and a role list, a relational constraint, and a predicate set are all count-shaped surfaces with the same defect available in each. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 75 ++++++++++++++++++-- 1 file changed, 68 insertions(+), 7 deletions(-) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index f534d7c1b4..d16d502557 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -157,9 +157,16 @@ Three additional legs close gaps a single-target exit-code test leaves open: - **Client readiness.** A boundary with no working TLS client would otherwise "pass" trivially — a missing tool is indistinguishable from a denied network. The same client must first succeed against an in-boundary endpoint; a client that cannot be shown to run yields UNPROVEN, never a pass. -- **Multiple targets, one unguessable.** One denied destination does not establish default-deny: a - policy can allow specific hosts while denying the probed one. At least two targets are required, one - of them a randomized name no allowlist could have anticipated. +- **Multiple targets under different operators.** One denied destination does not establish + default-deny: a policy can allow specific hosts while denying the probed one. At least two DISTINCT + external targets are required, each independently proven reachable from the outer context. + + *Scope change, 2026-08-10, found at implementation.* ~~one of them a randomized name no allowlist + could have anticipated~~ — a randomized name cannot satisfy the recipe's existing outer-reachability + invariant (`outer_exit_code` must be `"0"`, because a target that fails everywhere "fails" inside too + and proves nothing). Unguessability and outer-reachability are mutually exclusive over DNS, and + outer-reachability is the older, load-bearing rule. Distinct operators is the nearest property that + survives it. - **Address family recorded, never inferred.** IPv4 and IPv6 are probed and recorded separately; an absent family is `not-applicable`, never counted as denied. @@ -204,7 +211,49 @@ recorded as a deferred item with its own trigger rather than implied by the asse Phase order is dependency-driven: the vocabulary leaf (Phase 2) must exist before any surface can cite it, and Phase 1 is independent of every other phase. -#### Phase 1: Probe hardening — data-flow egress + workspace host-write containment [TODO] +#### Phase 1: Probe hardening — data-flow egress + workspace host-write containment [DONE] + +**Merge gate satisfied 2026-08-10.** Evidence: `.work/docker-sandbox-substrate/probe-evidence-hardened-recipe.md`. +All three assertions failed inside a live boundary; 446/446 fixture checks pass; 0 pre-existing pinned +reasons changed. + +**What the live run changed.** It was not a formality — it found two recipe defects and one stale +environment claim that no amount of review would have caught: + +- **The peer-identity design is empirically vindicated, not merely reasoned.** The measured boundary + presented a certificate with the CORRECT hostname signed by a CA it trusted, so + `ssl_verify_result=0` — certificate verification returned SUCCESS on a fully sealed boundary. Only + the differing fingerprint distinguished interception from reached egress. This substrate is a live + instance of the TLS-inspection case, not a hypothetical one. +- **A direct-TLS fingerprint tool cannot traverse an HTTP `CONNECT` proxy** and reports no peer at + all, identically for a sealed and an open boundary. The recipe now requires a proxy-aware capture. +- **The probe shape now shows fail-on-HTTP-error explicitly.** The first attempt at the live run + reproduced the original false negative exactly — a block page is a successful transfer. +- **Handoff correction:** `policy ls` does not display the global network policy, so the posture reads + as absent. `policy check network ` is the confirmation route. + +**PR review outcome (#2150).** Two reviewers — one cross-vendor, one same-vendor — independently +converged on the SAME five findings, which is what marked them real rather than stylistic. All five +were confirmed and fixed; 474 checks pass, 0 pre-existing pinned reasons changed. + +- **One was a stale-justification failure worth carrying forward as a lesson.** The plan recorded that + the non-zero-exit invariant "stands unchanged — no relaxation", justified by the claim that + certificate verification achieved the same discrimination while keeping the exit requirement. + Certificate verification was then REPLACED by peer-identity comparison, which destroyed that + justification, but the exit requirement survived unexamined — leaving a sealed boundary whose block + page carries a successful HTTP status unprovable. A zero exit is now accepted only where the entry's + outcome is `peer-substituted` and its fingerprints differ. **When a decision's premise is replaced, + re-derive every rule that premise was carrying.** +- **The other four shared one shape: counting entries where coverage was meant.** Two egress entries + counted as two targets (`example.com,example.com` passed); three canary strings counted as three + shapes (`same,same,same` passed); fingerprints went unchecked on outcomes that assert no peer; + `",,"` counted as three recorded exit codes. **Phase 3 inherits this directly** — a role list, a + relational constraint, and a predicate set are all count-shaped surfaces, and the same defect is + available in each unless the checks assert distinctness and coverage rather than length. +- **Pushed back on one half-suggestion rather than complying:** the checker has no trustworthy way to + verify two hosts belong to different operators, since every signal it could read comes from the same + transcript an executing agent authored. The recipe states the requirement; the checker enforces the + verifiable part; the gap is recorded rather than implied. Review: security @@ -264,9 +313,21 @@ Criterion 3. Delivers the Q20 resolution above. - `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with every pre-existing case's `findings_substrings` unchanged (diff the manifest: only ADDED keys, no MODIFIED `findings_substrings` on pre-existing cases). -- `grep -c "workspace_host_write_contained" plugins/autonomy/skills/setup/templates/isolation-probe.md` ≥ 3. -- `grep -rn "must fail to CONNECT\|SAME two assertions\|Two checks\|both assertions" plugins/autonomy/skills/setup/` - returns empty. +- The template documents the assertion in both places that matter, asserted precisely rather than by + token count: `grep -c "Workspace host-write containment probe shape" templates/isolation-probe.md` + returns 1, and `grep -c "workspace_host_write_contained" templates/isolation-probe.md` returns ≥ 1 + (the transcript capture shape). + + *Correction, 2026-08-10, found at implementation.* The original `≥ 3` literal-token count was + arbitrary and failed against a template that documents the assertion correctly in prose. Padding the + document to satisfy the count would have been the wrong repair. +- `grep -rn "SAME two assertions\|Two checks\|both assertions\|two assertions" plugins/autonomy/skills/setup/` + returns empty — every hardcoded assertion COUNT moved with the assertion set. + + *Correction, 2026-08-10, found at implementation.* ~~`must fail to CONNECT`~~ was wrongly included + in this zero-match list. That phrase belongs to the CREDENTIAL assertion's metadata-endpoint clause, + where connection-level failure is still the correct requirement; only the EGRESS assertion changed. + Deleting it would have damaged sound contract text to satisfy a bad check. - `grep -rn "sbx\|Docker Sandboxes\|Multipass\|Hyper-V" plugins/autonomy/` returns empty. - A re-probe transcript exists under `.work/docker-sandbox-substrate/` recording the new `transport_outcome` and `workspace_host_write_contained` blocks from a live run. From 22c7ffab5b35c87da75dcf08bf367fbb719ffe54 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Mon, 10 Aug 2026 11:33:41 -0400 Subject: [PATCH 06/17] docs(topics): correct the policy-listing claim that cleanup falsified MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The note asserted that the substrate's policy listing never displays the global network policy. Removing both probe sandboxes showed otherwise: the listing displayed the deny-all posture plainly once the kit-scoped policies were gone. What was actually observed is narrower — the global policy was absent from the listing while those kit policies existed — and the mechanism is unexplained rather than asserted. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index d16d502557..8e3f96aa96 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -229,8 +229,13 @@ environment claim that no amount of review would have caught: all, identically for a sealed and an open boundary. The recipe now requires a proxy-aware capture. - **The probe shape now shows fail-on-HTTP-error explicitly.** The first attempt at the live run reproduced the original false negative exactly — a block page is a successful transfer. -- **Handoff correction:** `policy ls` does not display the global network policy, so the posture reads - as absent. `policy check network ` is the confirmation route. +- **Handoff correction, itself corrected during cleanup.** The global deny-all posture did NOT appear + in `policy ls` while two kit-scoped policies were present, which is what made it read as absent — and + the first version of this note wrongly concluded that the listing never displays the global policy. + Once both sandboxes were removed, the listing showed `default-deny-all` plainly. The mechanism behind + the earlier omission is unexplained and is NOT claimed here. What holds either way: + **`policy check network ` is the authoritative confirmation route**, because it reports the + effective decision rather than the listing's view of it. **PR review outcome (#2150).** Two reviewers — one cross-vendor, one same-vendor — independently converged on the SAME five findings, which is what marked them real rather than stylistic. All five From e5f7f39acff0844f84811a5828b2859e4d35c783 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 17:54:34 -0400 Subject: [PATCH 07/17] docs(topics): resolve the three reserved decisions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Q21 was settled empirically rather than argued. The premise — that an installed component can widen egress past a global default-deny — was confirmed only as a rule's EXISTENCE, never its effect, so it was tested: the component-allowed host returned 200 with the origin's own CA on the wire and 201,961 bytes read, while the control host returned a 403 block page behind the interceptor's CA. The concern is proven, not suspected. The decision is a class-level property rather than a rule naming the vendor's component type: an L2+ binding must assert that nothing the run can install can widen the boundary, only narrow it. That generalizes to any additive policy engine, and it survives the ladder's classes-never-vendors rule, which the alternative would not. It also creates a probe obligation that is easy to miss. Target selection is load-bearing: a probe sampling only hosts the installed components do not allow will certify a boundary that is in fact open. Q23's framing did not survive its own corpus. Release cadence is evidentially inert in both directions, so re-verification binds to events that could change the probed property, with the repository's existing two-month idiom as a staleness backstop rather than a new number. Q22 drafts the three triggers in the runner charter's idiom for review. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 76 ++++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index 8e3f96aa96..d89f9c3720 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -584,6 +584,82 @@ Test-first throughout — the fixture harness is already the red-green loop for own release note. Phases 2–5 ship as a second PR carrying the topology change coherently. Alternative: one PR per phase, five review round-trips. +### Reserved decisions — RESOLVED 2026-08-11 + +All three were USER-RESERVED through planning and Phase 1. Research reframed two of them; the third +was settled empirically rather than argued. Evidence: +`.work/docker-sandbox-substrate/RESEARCH-reserved-questions.md`. + +#### Q21 — the ladder gains a class-level property, not a vendor-shaped rule + +**Decision: an `L2`+ binding must assert that nothing the run can install is able to WIDEN the +boundary — only narrow it.** + +The vendor already solves this under organization governance, where the documented precedence is +`kit allow ✗ / kit deny ✓` — *"Precedence is decided by a rule's decision rather than its source."* +It does NOT solve it in local-policy-only mode, which the vendor's table leaves unstated and which +this session measured: a component installed at sandbox-create time widened egress past a global +default-deny, and **201,961 bytes of origin data flowed**, with the origin's own CA on the wire. + +Naming the vendor's component type here would violate the ladder's classes-never-vendors rule, and +would not generalize. The property does: it covers browser-extension permissions, admission +controllers, and any additive policy engine. + +Work items (land with the Phases 2–5 PR, since Phase 1 already edits this file): + +- [ ] `reference/guardrails/isolation-ladder.md` — state the property on `L2`. An additive policy + engine whose components can only narrow satisfies it; one where an installed component can widen + does not, and that surface fails closed until governance is configured so it cannot. +- [ ] `templates/isolation-probe.md` — **the probe obligation this creates.** Target selection is + load-bearing: a probe sampling only hosts the installed components do NOT allow will certify a + boundary that is in fact open. The recipe must require probing in the configuration the run will + actually use, with at least one target drawn from what the installed components are permitted to + reach. +- [ ] Fixture: a transcript whose probed targets exclude every component-allowed host is not a + conforming capture. + +#### Q22 — the three software-factory gaps get triggers in the T4 idiom + +**Decision: drafted below in the runner charter's trigger idiom — a named, judgement-free condition, +explicitly not assumed to have fired.** Wording is for review. + +- **Fleet-level economics as tracked output.** *Trigger:* the return-accounting and telemetry contracts + are both bound and emitting for more than one repository under one org binding, AND a question is + asked of that data which per-run records cannot answer (cost or yield compared ACROSS repositories). + Until then the existing per-run contracts cover the need and a fleet aggregate would have no second + repository to aggregate. +- **Portfolio-scale multi-repo fan-out as a unit of work.** *Trigger:* a single work item requires + coordinated change across two or more repositories with a shared acceptance criterion, and the + per-run seams cannot express it without a human sequencing the runs. Until then every seam is + per-run by construction and fan-out has no unit to carry. +- **Self-service golden paths for humans and agents.** *Trigger:* a second adopter (any consumer + outside the authoring org) completes guided setup, OR the setup interview's unanswered-value rate + makes the interview itself the bottleneck. Until then a golden path would be generalized from a + single deployment, which is the sample size this repository already rejects elsewhere. + +Each is recorded as DEFERRED WITH A TRIGGER, never as rejected — the ladder's own "Rejected axis" +section is reserved for what was deliberately not chosen, which these are not. + +#### Q23 — event-triggered re-verification, with a staleness bound as backstop + +**Decision: re-verify on events that could change the probed property; cap evidence age separately. +No cadence keyed to release frequency.** + +The corpus already rejected the framing the question assumed. Release cadence is *evidentially inert*: +CISA warns against reading fix counts as a negative signal; Ozment & Schechter measure median +foundational vulnerability lifetime at **≥2.6 years** with ~67.6% found after 7.5; Rescorla cannot +exclude a constant discovery rate. Three weeks is not a sample, so a release-frequency cadence would +be ritual rather than control. + +- **Re-verification events:** a substrate version change touching the probed boundary; a policy-engine + or governance-mode change; a change to the installed component set (which Q21 just proved can widen + the boundary without any version change at all). +- **Staleness bound:** evidence older than the repository's existing **">2-month"** idiom is stale + regardless of events. Reusing that number rather than inventing one — it is already the corpus's + own gate. +- **What re-verification covers:** the specifics that rot — flags, defaults, guarantees. The + architecture and posture findings are durable and are not re-derived each time. + ### Deferred, with triggers — recorded so they are not silently implied - **Workspace READ-exposure assertion.** Trigger: any adopter binds a substrate whose workspace mount From 5af9218cc2c744739b6a175e31b6a060afbc8303 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 19:34:04 -0400 Subject: [PATCH 08/17] docs(topics): mark Phase 1 shipped and carry its unfinished obligation forward Records the merge, and states plainly what Phase 1 did NOT close: the probe obligation the Q21 decision creates. Target selection is load-bearing and the shipped recipe does not yet require a target drawn from what the run's installed components may reach, so a conforming transcript can still certify an open boundary. Measured at 201,961 bytes across a global default-deny. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index d89f9c3720..b331277996 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -213,9 +213,17 @@ cite it, and Phase 1 is independent of every other phase. #### Phase 1: Probe hardening — data-flow egress + workspace host-write containment [DONE] +**SHIPPED.** PR #2150 merged as `c1b4c629`; `autonomy` is at `0.15.0` on `main`. 34/34 checks, +11/11 review threads resolved, 474 fixture checks, 0 pre-existing pinned reasons changed. + **Merge gate satisfied 2026-08-10.** Evidence: `.work/docker-sandbox-substrate/probe-evidence-hardened-recipe.md`. -All three assertions failed inside a live boundary; 446/446 fixture checks pass; 0 pre-existing pinned -reasons changed. +All three assertions failed inside a live boundary. + +**Carried into Phase 2 — Phase 1 does NOT satisfy the Q21 probe obligation.** Target selection is +load-bearing, and the shipped recipe does not yet require a target drawn from what the run's installed +components may reach. Measured consequence: 201,961 bytes of origin data crossed a global +default-deny through a component-installed allow rule. Until that leg lands, a conforming transcript +can still certify a boundary that is open. **What the live run changed.** It was not a formality — it found two recipe defects and one stale environment claim that no amount of review would have caught: From c86752e22c0059a51f60886d0e3f36f0a4cc2d3e Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 19:56:50 -0400 Subject: [PATCH 09/17] docs(autonomy): add the verification-topology contract leaf MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Verification policy so far had no vocabulary for WHO verifies a change. The security-review leaf owns which layers exist and which gate a merge; nothing owned the population that runs them, so "a second fresh run checks the implementation, a third does the code review" was expressible only as prose. The leaf fixes that vocabulary: roles that are properties of this pipeline rather than any vendor's roster, relational constraints that bind a role by its relationship to another role, and predicates a binding can actually EVALUATE. A requirement that cannot be evaluated is a preference, and preferences are not policy — which is why not_weaker_than ships present but not defaulted: no cross-vendor capability ordering exists to evaluate it against. min_checkers counts DISTINCT checker roles, never repeated runs of one instance. N runs of one instance share the failure the count exists to catch, and stating that in the contract is what keeps the later schema check from asserting length where it means coverage. Capability labels are recorded as rejected as policy vocabulary, with no such label written anywhere in the contract — not even as a rejected example, since a rejected-vocabulary section still puts the words in the normative artifact. The sourced per-label evidence stays in the change record. Two invariants ship as invariants rather than knobs: independent aggregation never deliberation, and unanimous checker agreement for any transition a run takes without a human. The hub gains the column, its one-line definition, and the routing row. Its "five guardrail columns" sentence moves with the column set. One pre-existing capability label in the pipeline-reminder's provenance note is dropped: the plugin README owns that citation, so the label was carrying no identification the contract surface is allowed to carry. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- .../reference/autonomous-pipeline-reminder.md | 2 +- plugins/autonomy/reference/guardrails.md | 20 ++-- .../guardrails/verification-topology.md | 100 ++++++++++++++++++ 3 files changed, 113 insertions(+), 9 deletions(-) create mode 100644 plugins/autonomy/reference/guardrails/verification-topology.md diff --git a/plugins/autonomy/reference/autonomous-pipeline-reminder.md b/plugins/autonomy/reference/autonomous-pipeline-reminder.md index f28dfb8672..1af7e7ef18 100644 --- a/plugins/autonomy/reference/autonomous-pipeline-reminder.md +++ b/plugins/autonomy/reference/autonomous-pipeline-reminder.md @@ -106,7 +106,7 @@ the rest — not a gate presented as if it covered all seven. ## Provenance The clause set is this repository's own wording of guidance published in a model vendor's prompting -guide for its frontier model. It is authored here rather than reproduced, per this repository's rule +guide. It is authored here rather than reproduced, per this repository's rule against hand-copying upstream content — so it is a locally-owned artifact that cannot silently drift out of sync with a copy, while the guide stays the thing to read when the upstream advice changes. diff --git a/plugins/autonomy/reference/guardrails.md b/plugins/autonomy/reference/guardrails.md index 5066582950..37c5517328 100644 --- a/plugins/autonomy/reference/guardrails.md +++ b/plugins/autonomy/reference/guardrails.md @@ -1,7 +1,7 @@ # Guardrails Normative contract for per-work-class guardrail enforcement: five semantic risk classes -(`C1`–`C5`) crossed with five guardrail columns, in one matrix. This document is the hub and +(`C1`–`C5`) crossed with six guardrail columns, in one matrix. This document is the hub and glance layer of a progressive-disclosure contract — the matrix plus the one-line class and column definitions below alone answer "what governs class X"; every deeper question routes to a named leaf under `guardrails/`, loaded on demand. Vocabulary is contract-owned; every @@ -16,13 +16,13 @@ guardrail axis. That sentence is the playbook's; the five-class taxonomy, the pe blocking knobs, and the promotion predicates that fill the cells are this contract's own instantiation of it — the playbook names the obligation, this contract supplies the mechanism. -| Class | Min isolation (unattended) | Verification | Merge policy | Cost tier | Escalation | -|---|---|---|---|---|---| -| C1 | L2 (exfil surface remains) | output-shape checks | n/a; artifacts via queue audit trail | economy | low | -| C2 | L2 | deterministic blocking | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | economy | gate failure → human | -| C3 | L2 | deterministic blocking + AI review (advisory, promotable per [the security-review policy](guardrails/security-review.md)) | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | standard | divergence/failed verify → human | -| C4 | L2 | deterministic + AI + human review mandatory | human merge always | premium | upfront plan approval | -| C5 | L3 | full gates + zero secret exposure | human merge always | standard | always | +| Class | Min isolation (unattended) | Verification | Verification topology | Merge policy | Cost tier | Escalation | +|---|---|---|---|---|---|---| +| C1 | L2 (exfil surface remains) | output-shape checks | 1 checker | n/a; artifacts via queue audit trail | economy | low | +| C2 | L2 | deterministic blocking | 1 checker | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | economy | gate failure → human | +| C3 | L2 | deterministic blocking + AI review (advisory, promotable per [the security-review policy](guardrails/security-review.md)) | 2 distinct checkers | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | standard | divergence/failed verify → human | +| C4 | L2 | deterministic + AI + human review mandatory | 3 distinct checkers, one cross-vendor | human merge always | premium | upfront plan approval | +| C5 | L3 | full gates + zero secret exposure | 3 distinct checkers, one cross-vendor | human merge always | standard | always | ### Classes @@ -40,6 +40,9 @@ instantiation of it — the playbook names the obligation, this contract supplie for running the class unattended. - **Verification** — the gate layers a change must pass, with per-layer blocking knobs bound on the governance surface. +- **Verification topology** — the floor for how many DISTINCT checker roles verify the class + and whether one must be cross-vendor; roles, relational constraints, and predicates are + contract vocabulary, instances org-bound. - **Merge policy** — who lands the change; promotion-gated where the cell says so. - **Cost tier** — contract vocabulary (`economy` | `standard` | `premium`); the org binds tiers to model instances. Policy vocabulary only — cost enforcement is out of scope. @@ -56,6 +59,7 @@ answered from this document: | What each level means, which substrate classes satisfy it, what happens when none does | [isolation-ladder](guardrails/isolation-ladder.md) | | What exactly is in each class, and how a promotable cell promotes or demotes | [work-classes](guardrails/work-classes.md) | | Which verification layers exist and which block, per class | [security-review](guardrails/security-review.md) | +| Which roles verify a class, how they must differ, and the shipped checker floors | [verification-topology](guardrails/verification-topology.md) | | Which signals may enter the queue autonomously, and under what caps | [admission-policy](guardrails/admission-policy.md) | ## Permission posture diff --git a/plugins/autonomy/reference/guardrails/verification-topology.md b/plugins/autonomy/reference/guardrails/verification-topology.md new file mode 100644 index 0000000000..2732b16efe --- /dev/null +++ b/plugins/autonomy/reference/guardrails/verification-topology.md @@ -0,0 +1,100 @@ +# Verification topology + +Normative leaf of the [guardrail contract](../guardrails.md): WHO verifies a change, how those +verifiers must differ from each other, and the per-class floor for how many there are. The +[security-review leaf](security-review.md) owns which verification LAYERS exist and which of them +gate a merge; this leaf owns the population that runs them. Roles, constraints, and predicates are +contract vocabulary; every concrete model instance is an org-binding outcome on the binding seam. + +## Roles + +Roles are properties of THIS pipeline, not of any vendor's roster, so a roster change never edits +policy — only the binding that resolves a role to an instance. + +| Role | Adjudicates | +|---|---| +| `generator` | produces the artifact under verification | +| `checker` | judges that artifact, in isolation from every other checker | +| `cross_vendor_checker` | a `checker` additionally constrained to a different vendor from the `generator` | +| `ranker` | orders candidates or findings relative to each other rather than scoring one absolutely | + +## Relational constraints + +A constraint binds a role by its relationship to another role, never by naming an instance. + +| Constraint | Resolves via | Why | +|---|---|---| +| `distinct_model_from: ` | model identity at run time | a model judging its own output measures its own preference, not the artifact | +| `distinct_vendor_from: ` | vendor identity | disjoint model families fail independently; same-vendor checkers share failure modes, so agreement between them is weaker evidence than its count suggests | +| `not_weaker_than: ` | an ordering source the binding declares | PRESENT BUT NOT DEFAULTED — no cross-vendor capability ordering exists to evaluate it against, so no shipped default uses it. A binding may state it only where it also declares its own ordering source | + +A constraint naming a role that its own class does not declare is invalid, not ignored. + +## Machine-checkable predicates + +A predicate is a requirement a binding can EVALUATE against a candidate instance. A requirement that +cannot be evaluated is a preference, and preferences are not policy. + +| Predicate | Resolution source | +|---|---| +| `min_context_tokens: N` | the declared input limit of the bound instance | +| `requires_modality: [...]` | the declared input/output modalities of the bound instance | +| `requires_feature: [...]` | the bound instance's declared feature set — feature NAMES are vendor-local, so the binding declares the mapping it resolves against | + +## Budget + +`max_input_cost_per_mtok` / `max_output_cost_per_mtok` — a per-role ceiling. This REFINES the +matrix's cost-tier column and never replaces it: the tier is the class-level cost vocabulary, the +ceiling is a numeric bound inside it. No vendor supplies a price feed, so the ceiling resolves +against an org-maintained table; a price that will not resolve leaves the role UNPROVEN and the +class's topology invalid, per the contract's uniform fail-closed rule. + +## Pins + +`pinned_model_id` is the one place a concrete instance identifier is legal, and only to reproduce a +RECORDED MEASUREMENT. Append-only: a recorded result keeps its pin forever, so pins accumulate and +never need updating. A pin never selects a role for new work and is never a policy default. + +## Rejected vocabulary: capability labels + +Capability labels — words naming how capable a model is, rather than what it must do or how it must +differ — are RECORDED AS REJECTED as policy vocabulary. Falsified twice over: each such word names a +different thing at each vendor, and none survives a model release. No such label appears anywhere in +this contract; a binding that introduces one is expressing preference where the contract requires a +resolvable constraint. The sourced per-label evidence lives in the change record that introduced +this leaf, deliberately outside the contract surface. + +## Shipped floors + +| Class | `min_checkers` | `cross_vendor_required` | +|---|---|---| +| `C1` | 1 | no | +| `C2` | 1 | no | +| `C3` | 2 | no | +| `C4` | 3 | yes | +| `C5` | 3 | yes | + +**`min_checkers` counts DISTINCT checker roles, each satisfying its own relational constraints — +never repeated runs of one instance.** N runs of a single instance count as one checker: they share +the failure the count exists to catch. + +Shipped values are FLOORS: a binding may tighten any cell but never weaken one below its shipped +value, and no justification field excuses a weakening — the same rule the security-review knobs +carry. Floors bind ONLY on the org's security governance surface, outside the blast radius of the +agents they govern; a floor those agents can lower is no floor. An absent or invalid binding +fail-closes to the shipped values above. + +## Two fixed invariants + +Neither is a knob, and no binding may relax either. + +**Independent aggregation, never deliberation.** Checkers run isolated: no checker sees another +checker's verdict or reasoning, and verdicts are combined mechanically. Deliberation between +checkers is RECORDED AS REJECTED — falsified: a measured deliberative protocol scored BELOW every +single-model baseline it was built from, while independent aggregation over the same models scored +above them. Agreement reached by discussion is correlation, not corroboration. + +**Unanimous checker agreement for anything auto-proceeding.** Every transition a run takes without a +human — not merge alone — requires every checker the class declares to agree. One dissent withholds +the automatic transition and hands the item to the human gate; divergence routing is owned by the +matrix's escalation contract. From 8ce7cd463ec87bad35eac82f7b2bb8f915106df4 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 20:36:23 -0400 Subject: [PATCH 10/17] fix(autonomy): make the checker floor a predicate a binding can evaluate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An independent audit of the topology leaf found its headline number was not machine-checkable, which is the half of the criterion the label grep does not test. The Roles table names role TYPES, only one of which is `checker`, so "3 distinct checker roles" either could not be satisfied or meant list length — and a class declaring [A, A, B] passed both the count and the cross-vendor requirement. That is the same count-where-coverage-was-meant defect a prior review round caught four times in this effort, reproduced in the artifact written to prevent it. Slots are now the counted unit, distinctness is implied on every slot rather than opted into, and the rule is stated over the binding rather than over runs — a validator reads a binding, so a rule about runs is unevaluable by construction. Two slots that resolve identically declare one checker. Slots also now admit either a deterministic layer or a model-adjudicated role. Without that split the C1 and C2 floors asserted a model judge whose own verification cells say none is required, since every mechanism in the leaf presumes a model instance. The floors now compose with the security-review layers instead of contradicting them, and the human review C4 mandates is stated as the merge gate rather than left to be guessed at as a slot. `cross_vendor_required` obliged nothing its own rationale argued for: it constrained checkers against the generator only, so three checkers sharing one vendor satisfied it while sharing every failure mode the constraint exists to break up. It now obliges disjointness among the checkers too. Three pointers claimed support that did not exist. The budget ceiling was written as invalidating, which would have made this leaf the single enforcing exception to the matrix's own out-of-scope statement on cost and to the roadmap trigger gating spend caps — it is recorded, not enforcing. A uniform fail-closed rule was cited that no file states, and the leaf already states its own. A justification-field clause was attributed to the security-review knobs, which do not carry it. The deliberation invariant rested on an unnamed measurement with no figures and no citation — strictly worse than a parked number, since a reader can neither confirm nor rebut it. It now rests on the mechanism alone, which carries it honestly; the measurement and its confidence grade move to the issue. The plugin README still described five enforcement columns and listed neither the new column nor the new leaf — the hub sentence moved and its mirror did not. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- plugins/autonomy/README.md | 7 ++- .../reference/autonomous-pipeline-reminder.md | 6 +- plugins/autonomy/reference/guardrails.md | 9 +-- .../guardrails/verification-topology.md | 59 ++++++++++++++----- 4 files changed, 55 insertions(+), 26 deletions(-) diff --git a/plugins/autonomy/README.md b/plugins/autonomy/README.md index b619ab3121..f2b64da79c 100644 --- a/plugins/autonomy/README.md +++ b/plugins/autonomy/README.md @@ -26,9 +26,10 @@ state and records that binding. lease) — plus the setup trigger/dispatch slice, its adapter and acknowledgment templates, and the signal-envelope conformance check. - **Guardrail matrix** (`reference/guardrails.md`): five semantic work classes (`C1`–`C5`) - crossed with five enforcement columns — isolation floor, verification layers, merge policy, - cost tier, escalation — as one progressive-disclosure hub with on-demand leaves (isolation - ladder, work classes, security review, admission policy), human-ratified promotion with + crossed with six enforcement columns — isolation floor, verification layers, verification + topology, merge policy, cost tier, escalation — as one progressive-disclosure hub with + on-demand leaves (isolation ladder, work classes, security review, verification topology, + admission policy), human-ratified promotion with automatic fail-closed demotion, and a two-surface binding split by governance sensitivity (security axes on the settings-as-code home outside agent blast radius; non-security remaps repo-local) — plus the contract-owned security-binding schema and its semantic check, and the diff --git a/plugins/autonomy/reference/autonomous-pipeline-reminder.md b/plugins/autonomy/reference/autonomous-pipeline-reminder.md index 1af7e7ef18..24d0782bc7 100644 --- a/plugins/autonomy/reference/autonomous-pipeline-reminder.md +++ b/plugins/autonomy/reference/autonomous-pipeline-reminder.md @@ -106,9 +106,9 @@ the rest — not a gate presented as if it covered all seven. ## Provenance The clause set is this repository's own wording of guidance published in a model vendor's prompting -guide. It is authored here rather than reproduced, per this repository's rule -against hand-copying upstream content — so it is a locally-owned artifact that cannot silently drift -out of sync with a copy, while the guide stays the thing to read when the upstream advice changes. +guide. It is authored here rather than reproduced, per this repository's rule against hand-copying +upstream content — so it is a locally-owned artifact that cannot silently drift out of sync with a +copy, while the guide stays the thing to read when the upstream advice changes. **The citation, the exact section, and the recheck trigger live in the plugin [`README.md`](../README.md), not here.** These `reference/` contracts are written in surface classes diff --git a/plugins/autonomy/reference/guardrails.md b/plugins/autonomy/reference/guardrails.md index 37c5517328..40914ddf7a 100644 --- a/plugins/autonomy/reference/guardrails.md +++ b/plugins/autonomy/reference/guardrails.md @@ -40,9 +40,10 @@ instantiation of it — the playbook names the obligation, this contract supplie for running the class unattended. - **Verification** — the gate layers a change must pass, with per-layer blocking knobs bound on the governance surface. -- **Verification topology** — the floor for how many DISTINCT checker roles verify the class - and whether one must be cross-vendor; roles, relational constraints, and predicates are - contract vocabulary, instances org-bound. +- **Verification topology** — the floor for how many DISTINCT checker slots verify the class + and whether they must be vendor-disjoint; roles, relational constraints, and predicates are + contract vocabulary, instances org-bound. The leaf owns these floor values; the cells above + are their glance restatement. - **Merge policy** — who lands the change; promotion-gated where the cell says so. - **Cost tier** — contract vocabulary (`economy` | `standard` | `premium`); the org binds tiers to model instances. Policy vocabulary only — cost enforcement is out of scope. @@ -76,7 +77,7 @@ Six escalation event classes: | Event class | Fires when | |---|---| | `gate-failure` | a blocking verification gate fails | -| `verification-divergence` | a verification outcome diverges from the expected or claimed result | +| `verification-divergence` | a verification outcome diverges from the expected or claimed result, or two checkers of one class disagree | | `admission-rejection` | the admission seam rejects a signal as an audited rejection | | `demotion` | contrary evidence automatically demotes a promoted cell | | `structural-plan-approval` | a `C4` item requires upfront plan approval before execution | diff --git a/plugins/autonomy/reference/guardrails/verification-topology.md b/plugins/autonomy/reference/guardrails/verification-topology.md index 2732b16efe..fe29f693df 100644 --- a/plugins/autonomy/reference/guardrails/verification-topology.md +++ b/plugins/autonomy/reference/guardrails/verification-topology.md @@ -15,9 +15,25 @@ policy — only the binding that resolves a role to an instance. |---|---| | `generator` | produces the artifact under verification | | `checker` | judges that artifact, in isolation from every other checker | -| `cross_vendor_checker` | a `checker` additionally constrained to a different vendor from the `generator` | +| `cross_vendor_checker` | a `checker` additionally constrained to a vendor shared with no other checker and not the `generator`'s | | `ranker` | orders candidates or findings relative to each other rather than scoring one absolutely | +## Checker slots + +A class's topology declares a list of checker SLOTS, and a slot is filled by either a DETERMINISTIC +layer or a MODEL-ADJUDICATED role. The distinction is load-bearing: a deterministic layer has no +model or vendor identity, so the relational constraints and predicates below bind only +model-adjudicated slots and are never required of a deterministic one. + +**Distinctness is implied on every slot — it is not something a binding opts into.** Two slots are +distinct only where they cannot share a failure mode: deterministic slots are distinguished by +scanner class, model-adjudicated slots by resolved model identity. **Two slots that resolve +identically declare ONE checker**, and a binding whose distinct-slot count falls below its class +floor is invalid. The rule is stated over the BINDING rather than over runs, because a binding is +what a check can read. + +The human review the matrix makes mandatory for `C4` is NOT a checker slot. It is the merge gate. + ## Relational constraints A constraint binds a role by its relationship to another role, never by naming an instance. @@ -25,7 +41,7 @@ A constraint binds a role by its relationship to another role, never by naming a | Constraint | Resolves via | Why | |---|---|---| | `distinct_model_from: ` | model identity at run time | a model judging its own output measures its own preference, not the artifact | -| `distinct_vendor_from: ` | vendor identity | disjoint model families fail independently; same-vendor checkers share failure modes, so agreement between them is weaker evidence than its count suggests | +| `distinct_vendor_from: ` | vendor identity | disjoint model families fail independently; same-vendor checkers share failure modes, so agreement between them is weaker evidence than its count suggests. `cross_vendor_required` therefore obliges vendor disjointness AMONG the model-adjudicated slots as well as from the `generator` — a class whose checkers all share one vendor satisfies neither the constraint nor the reason for it | | `not_weaker_than: ` | an ordering source the binding declares | PRESENT BUT NOT DEFAULTED — no cross-vendor capability ordering exists to evaluate it against, so no shipped default uses it. A binding may state it only where it also declares its own ordering source | A constraint naming a role that its own class does not declare is invalid, not ignored. @@ -46,8 +62,12 @@ cannot be evaluated is a preference, and preferences are not policy. `max_input_cost_per_mtok` / `max_output_cost_per_mtok` — a per-role ceiling. This REFINES the matrix's cost-tier column and never replaces it: the tier is the class-level cost vocabulary, the ceiling is a numeric bound inside it. No vendor supplies a price feed, so the ceiling resolves -against an org-maintained table; a price that will not resolve leaves the role UNPROVEN and the -class's topology invalid, per the contract's uniform fail-closed rule. +against an org-maintained table. + +**The ceiling is RECORDED, never enforcing.** A price that will not resolve is recorded as +unresolved; it does not invalidate a binding and does not gate a run. The matrix states that cost +enforcement is out of scope, and hard spend caps are gated behind their own trigger — a ceiling that +blocked here would quietly make this leaf the one enforcing exception to both. ## Pins @@ -61,8 +81,8 @@ Capability labels — words naming how capable a model is, rather than what it m differ — are RECORDED AS REJECTED as policy vocabulary. Falsified twice over: each such word names a different thing at each vendor, and none survives a model release. No such label appears anywhere in this contract; a binding that introduces one is expressing preference where the contract requires a -resolvable constraint. The sourced per-label evidence lives in the change record that introduced -this leaf, deliberately outside the contract surface. +resolvable constraint. The sourced per-label evidence lives in the pull request that introduced this +leaf and on the issue it closes, deliberately outside the contract surface. ## Shipped floors @@ -74,15 +94,21 @@ this leaf, deliberately outside the contract surface. | `C4` | 3 | yes | | `C5` | 3 | yes | -**`min_checkers` counts DISTINCT checker roles, each satisfying its own relational constraints — -never repeated runs of one instance.** N runs of a single instance count as one checker: they share -the failure the count exists to catch. +`min_checkers` counts DISTINCT slots per the rule above. It is a coverage floor, never a list +length: a class declaring its floor count of slots that resolve identically has declared one +checker, and its binding is invalid. + +The floors compose with the [security-review layers](security-review.md) rather than duplicating +them — `C1`'s single slot is its output-shape check and `C2`'s is the deterministic scanner layer, +so neither class is obliged a model judge its verification cell does not require; `C3` adds one +model-adjudicated slot; `C4` and `C5` add two, vendor-disjoint. Shipped values are FLOORS: a binding may tighten any cell but never weaken one below its shipped -value, and no justification field excuses a weakening — the same rule the security-review knobs -carry. Floors bind ONLY on the org's security governance surface, outside the blast radius of the -agents they govern; a floor those agents can lower is no floor. An absent or invalid binding -fail-closes to the shipped values above. +value. The weakening-is-invalid rule is the security-review knobs' own; this leaf adds that no +justification field excuses a weakening either. Floors bind ONLY on the org's security governance +surface, outside the blast radius of the agents they govern; a floor those agents can lower is no +floor. An absent or invalid binding fail-closes to the shipped values above, which this leaf owns — +the matrix cells are their glance restatement. ## Two fixed invariants @@ -90,9 +116,10 @@ Neither is a knob, and no binding may relax either. **Independent aggregation, never deliberation.** Checkers run isolated: no checker sees another checker's verdict or reasoning, and verdicts are combined mechanically. Deliberation between -checkers is RECORDED AS REJECTED — falsified: a measured deliberative protocol scored BELOW every -single-model baseline it was built from, while independent aggregation over the same models scored -above them. Agreement reached by discussion is correlation, not corroboration. +checkers is RECORDED AS REJECTED — falsified: agreement reached by discussion is correlation, not +corroboration, so a deliberating panel converges toward whichever verdict is stated most +persuasively rather than toward the one the artifact supports, and the count of agreeing checkers +stops measuring independent confirmation the moment they can hear each other. **Unanimous checker agreement for anything auto-proceeding.** Every transition a run takes without a human — not merge alone — requires every checker the class declares to agree. One dissent withholds From e8fd38fe91ea55e8d5906d2fa1e4dde2a9011687 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 20:52:25 -0400 Subject: [PATCH 11/17] fix(autonomy): floor the model-adjudicated slots so the count implies its coverage MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-verification found the same count-vs-coverage shape one level down, in the split introduced to repair the previous one. Relations and predicates bind only model-adjudicated slots, so a C4 binding declaring three deterministic slots — secret detection, dependency audit, static analysis — met a floor of three, and cross_vendor_required then bound the empty set of model slots and was satisfied by declaring nothing. A binding was valid with no model judge at all, while the matrix cell mandates AI review and the security-review layer makes it blocking. A total count cannot express which KIND of coverage is owed, so the floors gain min_model_checkers alongside min_checkers, and cross_vendor_required is stated as never vacuously satisfiable: it requires at least two model-adjudicated slots, and a class asserting it with fewer is invalid rather than trivially conforming. Each class's composition is now absolute rather than a delta off an unnamed base, which admitted two readings that disagreed with the count. Distinctness was stated as implied, which a validator cannot act on: a slot NAME says nothing about what it resolves to. The binding now establishes it explicitly, and an undeclared constraint is the unevaluable case rather than a generous one. Identity equality is named as the floor of that test and not the whole of it — an alias, a reseller route, and adjacent versions of one family compare unequal while sharing every failure mode, so the contract states the requirement and the check enforces the part it can read. The deliberation clause is trimmed to the argument that is actually analytic. Its middle sentence predicted how a deliberating panel behaves, which is an empirical claim with no citation — worse than the measurement it replaced, since a reader could neither confirm nor rebut it. What remains needs no evidence: a count of agreeing checkers stops measuring independent confirmation the moment they can hear each other. The matrix cells still said "one cross-vendor" after the leaf moved to disjointness among all model slots, so the glance layer contradicted its own column definition twenty lines below and stated a weaker rule than the contract holds. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- plugins/autonomy/README.md | 4 +- plugins/autonomy/reference/guardrails.md | 6 +- .../guardrails/verification-topology.md | 67 ++++++++++++------- 3 files changed, 49 insertions(+), 28 deletions(-) diff --git a/plugins/autonomy/README.md b/plugins/autonomy/README.md index f2b64da79c..6c3dc66eb4 100644 --- a/plugins/autonomy/README.md +++ b/plugins/autonomy/README.md @@ -29,8 +29,8 @@ state and records that binding. crossed with six enforcement columns — isolation floor, verification layers, verification topology, merge policy, cost tier, escalation — as one progressive-disclosure hub with on-demand leaves (isolation ladder, work classes, security review, verification topology, - admission policy), human-ratified promotion with - automatic fail-closed demotion, and a two-surface binding split by governance sensitivity + admission policy), human-ratified promotion with automatic fail-closed demotion, and a + two-surface binding split by governance sensitivity (security axes on the settings-as-code home outside agent blast radius; non-security remaps repo-local) — plus the contract-owned security-binding schema and its semantic check, and the setup guardrail slice that detects substrates per surface, live-validates isolation with an diff --git a/plugins/autonomy/reference/guardrails.md b/plugins/autonomy/reference/guardrails.md index 40914ddf7a..a8854c43c0 100644 --- a/plugins/autonomy/reference/guardrails.md +++ b/plugins/autonomy/reference/guardrails.md @@ -20,9 +20,9 @@ instantiation of it — the playbook names the obligation, this contract supplie |---|---|---|---|---|---|---| | C1 | L2 (exfil surface remains) | output-shape checks | 1 checker | n/a; artifacts via queue audit trail | economy | low | | C2 | L2 | deterministic blocking | 1 checker | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | economy | gate failure → human | -| C3 | L2 | deterministic blocking + AI review (advisory, promotable per [the security-review policy](guardrails/security-review.md)) | 2 distinct checkers | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | standard | divergence/failed verify → human | -| C4 | L2 | deterministic + AI + human review mandatory | 3 distinct checkers, one cross-vendor | human merge always | premium | upfront plan approval | -| C5 | L3 | full gates + zero secret exposure | 3 distinct checkers, one cross-vendor | human merge always | standard | always | +| C3 | L2 | deterministic blocking + AI review (advisory, promotable per [the security-review policy](guardrails/security-review.md)) | 2 distinct checkers, 1 a model judge | auto-merge ELIGIBLE after per-class promotion trigger; ships human-gated | standard | divergence/failed verify → human | +| C4 | L2 | deterministic + AI + human review mandatory | 3 distinct checkers, 2 vendor-disjoint model judges | human merge always | premium | upfront plan approval | +| C5 | L3 | full gates + zero secret exposure | 3 distinct checkers, 2 vendor-disjoint model judges | human merge always | standard | always | ### Classes diff --git a/plugins/autonomy/reference/guardrails/verification-topology.md b/plugins/autonomy/reference/guardrails/verification-topology.md index fe29f693df..8f9b30087c 100644 --- a/plugins/autonomy/reference/guardrails/verification-topology.md +++ b/plugins/autonomy/reference/guardrails/verification-topology.md @@ -15,7 +15,7 @@ policy — only the binding that resolves a role to an instance. |---|---| | `generator` | produces the artifact under verification | | `checker` | judges that artifact, in isolation from every other checker | -| `cross_vendor_checker` | a `checker` additionally constrained to a vendor shared with no other checker and not the `generator`'s | +| `cross_vendor_checker` | the role a class uses to require vendor disjointness where its own floor does not — under `cross_vendor_required`, every model-adjudicated slot already carries it | | `ranker` | orders candidates or findings relative to each other rather than scoring one absolutely | ## Checker slots @@ -25,12 +25,23 @@ layer or a MODEL-ADJUDICATED role. The distinction is load-bearing: a determinis model or vendor identity, so the relational constraints and predicates below bind only model-adjudicated slots and are never required of a deterministic one. -**Distinctness is implied on every slot — it is not something a binding opts into.** Two slots are -distinct only where they cannot share a failure mode: deterministic slots are distinguished by -scanner class, model-adjudicated slots by resolved model identity. **Two slots that resolve -identically declare ONE checker**, and a binding whose distinct-slot count falls below its class -floor is invalid. The rule is stated over the BINDING rather than over runs, because a binding is -what a check can read. +**Distinctness is REQUIRED on every slot and cannot be opted out of.** Two slots are distinct only +where they cannot share a failure mode: deterministic slots are distinguished by scanner class, +model-adjudicated slots by resolved model identity. **Two slots that resolve identically declare ONE +checker**, and a binding whose distinct-slot count falls below its class floor is invalid. + +A slot NAME tells a validator nothing about what the slot resolves to, so distinctness that is only +intended is not distinctness. **The binding establishes it explicitly**: every model-adjudicated +checker slot declares `distinct_model_from` against the `generator` AND against every other checker +slot in its class. A binding that leaves it undeclared has not established it and is invalid — an +undeclared constraint is the unevaluable case, which is the same failure as declaring none. + +Identity equality is the FLOOR of that test, not the whole of it. Two identifiers can name one +underlying model — an alias, a route through a reseller, adjacent versions of one family — and those +share every failure mode while comparing unequal. **A binding declaring two slots it knows resolve +to the same underlying model has declared one checker.** A check cannot see that, so the contract +states the requirement and a check enforces the part it can read; the gap is recorded here rather +than implied away. The human review the matrix makes mandatory for `C4` is NOT a checker slot. It is the merge gate. @@ -40,7 +51,7 @@ A constraint binds a role by its relationship to another role, never by naming a | Constraint | Resolves via | Why | |---|---|---| -| `distinct_model_from: ` | model identity at run time | a model judging its own output measures its own preference, not the artifact | +| `distinct_model_from: ` | the model identity the binding declares — static, because a check reads a binding | a model judging its own output measures its own preference, not the artifact | | `distinct_vendor_from: ` | vendor identity | disjoint model families fail independently; same-vendor checkers share failure modes, so agreement between them is weaker evidence than its count suggests. `cross_vendor_required` therefore obliges vendor disjointness AMONG the model-adjudicated slots as well as from the `generator` — a class whose checkers all share one vendor satisfies neither the constraint nor the reason for it | | `not_weaker_than: ` | an ordering source the binding declares | PRESENT BUT NOT DEFAULTED — no cross-vendor capability ordering exists to evaluate it against, so no shipped default uses it. A binding may state it only where it also declares its own ordering source | @@ -86,22 +97,32 @@ leaf and on the issue it closes, deliberately outside the contract surface. ## Shipped floors -| Class | `min_checkers` | `cross_vendor_required` | -|---|---|---| -| `C1` | 1 | no | -| `C2` | 1 | no | -| `C3` | 2 | no | -| `C4` | 3 | yes | -| `C5` | 3 | yes | +| Class | `min_checkers` | `min_model_checkers` | `cross_vendor_required` | +|---|---|---|---| +| `C1` | 1 | 0 | no | +| `C2` | 1 | 0 | no | +| `C3` | 2 | 1 | no | +| `C4` | 3 | 2 | yes | +| `C5` | 3 | 2 | yes | `min_checkers` counts DISTINCT slots per the rule above. It is a coverage floor, never a list length: a class declaring its floor count of slots that resolve identically has declared one checker, and its binding is invalid. -The floors compose with the [security-review layers](security-review.md) rather than duplicating -them — `C1`'s single slot is its output-shape check and `C2`'s is the deterministic scanner layer, -so neither class is obliged a model judge its verification cell does not require; `C3` adds one -model-adjudicated slot; `C4` and `C5` add two, vendor-disjoint. +**`min_model_checkers` exists because a total count cannot express which KIND of coverage is +owed.** Without it, a class meets its floor with deterministic slots alone and never faces a model +judge — and `cross_vendor_required` then binds an empty set and is satisfied by declaring nothing. +So it is never vacuously satisfied: **`cross_vendor_required: yes` requires at least two +model-adjudicated slots, pairwise vendor-disjoint and disjoint from the `generator`**, and a class +asserting it with fewer is invalid rather than trivially conforming. + +Each class's composition is absolute, stated against the +[security-review layers](security-review.md) it must not contradict: + +- `C1` — one slot: the output-shape check. +- `C2` — one slot: the deterministic scanner layer. +- `C3` — two slots: one deterministic layer and one model judge. +- `C4` and `C5` — three slots: one deterministic layer and two model judges, vendor-disjoint. Shipped values are FLOORS: a binding may tighten any cell but never weaken one below its shipped value. The weakening-is-invalid rule is the security-review knobs' own; this leaf adds that no @@ -116,10 +137,10 @@ Neither is a knob, and no binding may relax either. **Independent aggregation, never deliberation.** Checkers run isolated: no checker sees another checker's verdict or reasoning, and verdicts are combined mechanically. Deliberation between -checkers is RECORDED AS REJECTED — falsified: agreement reached by discussion is correlation, not -corroboration, so a deliberating panel converges toward whichever verdict is stated most -persuasively rather than toward the one the artifact supports, and the count of agreeing checkers -stops measuring independent confirmation the moment they can hear each other. +checkers is RECORDED AS REJECTED: agreement reached by discussion is correlation, not corroboration +— the count of agreeing checkers stops measuring independent confirmation the moment they can hear +each other, so a deliberating panel's unanimity means strictly less than an isolated panel's while +reading as if it meant more. **Unanimous checker agreement for anything auto-proceeding.** Every transition a run takes without a human — not merge alone — requires every checker the class declares to agree. One dissent withholds From 691272466dab487ecaf358ab6d00fe8b624aae79 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 21:48:51 -0400 Subject: [PATCH 12/17] feat(autonomy): enforce per-class verification topology on the security binding MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The topology leaf fixed the vocabulary; nothing yet made a binding express it or a checker reject one that did not. Verification policy was documentation, and a floor no binding can express is not a floor. verification_topology lands as an optional top-level key modeling all three axes, not only the count — a predicate that never reaches the binding is not machine-checkable, so the roles carry their relational constraints and predicates alongside the floors. Optional-additive with contract defaults follows the escalation_severity precedent, so schema_version stays 1.0 and every adopting org's binding keeps validating. An absent key is not a hole: the shipped floors apply. Every count-shaped surface here had the same defect available, the one four of five findings shared in the phase before it, so each is checked for coverage rather than length. Distinct role NAMES, not entry count. Distinct scanner classes for deterministic slots. Pairwise distinctness relations between model slots, because two checkers with nothing holding them apart can resolve to one instance and N runs of one instance are one checker. A relational constraint naming an undeclared role is invalid rather than ignored — that resolution is the whole difference between a machine-checkable predicate and a decorative one. min_model_checkers exists because a total count cannot express which KIND of coverage is owed. Without it a C4 binding met a floor of three with secret detection, dependency audit, and static analysis, faced no model judge at all, and satisfied cross_vendor_required over an empty set — valid while the matrix cell mandates AI review. So the knob is never vacuously satisfied: it requires at least two model-adjudicated slots, and vendor disjointness holds AMONG them rather than only against the generator, since three checkers sharing one vendor share every failure mode the requirement exists to break up. Deterministic and model-adjudicated slots are separated because the constraints cannot be evaluated against a scanner: it has no model or vendor identity, so declaring one on it is rejected rather than ignored. The pairwise distinctness check is scoped to model slots for the same reason — a cross-kind pair is distinct by construction, and requiring a relation there would demand a constraint the same checker forbids. Floors are floors: tightening legal, weakening invalid, no override_justification escape, matching the rule verification_blocking already carries. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- .../topology-all-deterministic-c4.json | 122 ++++++ .../topology-checker-count-below-min.json | 119 ++++++ .../topology-cross-vendor-decorative.json | 119 ++++++ .../topology-cross-vendor-dropped.json | 127 ++++++ .../topology-cross-vendor-no-generator.json | 112 ++++++ .../topology-cross-vendor-vacuous.json | 118 ++++++ .../topology-duplicate-modality.json | 123 ++++++ .../topology-duplicate-relation-target.json | 120 ++++++ .../topology-duplicate-role-name.json | 116 ++++++ .../topology-min-checkers-weakened.json | 117 ++++++ .../topology-non-class-key.json | 129 +++++++ .../topology-self-reference.json | 119 ++++++ .../topology-tightened-valid.json | 148 +++++++ .../topology-unconstrained-checker-pair.json | 116 ++++++ .../topology-undeclared-role.json | 117 ++++++ .../topology-unknown-predicate.json | 120 ++++++ .../guardrails-security-binding.schema.json | 82 ++++ ...curity-binding.fixtures.test.manifest.json | 112 ++++++ .../setup/scripts/check-security-binding.mjs | 364 +++++++++++++++++- 19 files changed, 2499 insertions(+), 1 deletion(-) create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json new file mode 100644 index 0000000000..5c5c7da77c --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json @@ -0,0 +1,122 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C4": { + "min_checkers": 3, + "min_model_checkers": 0, + "cross_vendor_required": true, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "secrets", + "role": "checker", + "scanner_class": "secret-detection" + }, + { + "name": "deps", + "role": "checker", + "scanner_class": "dependency-audit" + }, + { + "name": "sast", + "role": "checker", + "scanner_class": "static-analysis" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json new file mode 100644 index 0000000000..d44e5d8244 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json @@ -0,0 +1,119 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 3, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json new file mode 100644 index 0000000000..3420c0755e --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json @@ -0,0 +1,119 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": true, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json new file mode 100644 index 0000000000..c91db0021c --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json @@ -0,0 +1,127 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C5": { + "min_checkers": 3, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + }, + { + "name": "reviewer-c", + "role": "checker", + "distinct_model_from": [ + "reviewer-a", + "reviewer-b" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json new file mode 100644 index 0000000000..e2062fc2fe --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json @@ -0,0 +1,112 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": true, + "roles": [ + { + "name": "reviewer-a", + "role": "cross_vendor_checker", + "distinct_model_from": "reviewer-b" + }, + { + "name": "reviewer-b", + "role": "cross_vendor_checker", + "distinct_model_from": "reviewer-a" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json new file mode 100644 index 0000000000..8fb3beb1b3 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json @@ -0,0 +1,118 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "min_model_checkers": 1, + "cross_vendor_required": true, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "scanner", + "role": "checker", + "scanner_class": "secret-detection" + }, + { + "name": "reviewer-a", + "role": "cross_vendor_checker", + "distinct_model_from": "author", + "distinct_vendor_from": "author" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json new file mode 100644 index 0000000000..be09739665 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json @@ -0,0 +1,123 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author", + "requires_modality": [ + "text", + "text" + ] + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json new file mode 100644 index 0000000000..7fc2b3472a --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json @@ -0,0 +1,120 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "author" + ], + "distinct_vendor_from": "reviewer-a" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json new file mode 100644 index 0000000000..ecfc20802e --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json @@ -0,0 +1,116 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json new file mode 100644 index 0000000000..3d0a0726d0 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json @@ -0,0 +1,117 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C4": { + "min_checkers": 2, + "cross_vendor_required": true, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-c", + "role": "cross_vendor_checker", + "distinct_vendor_from": "author", + "distinct_model_from": "reviewer-a" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json new file mode 100644 index 0000000000..d9267bc7ae --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json @@ -0,0 +1,129 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + } + ] + }, + "C6": { + "min_checkers": 1, + "cross_vendor_required": false, + "roles": [ + { + "name": "reviewer-a", + "role": "checker" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json new file mode 100644 index 0000000000..8d96f151c1 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json @@ -0,0 +1,119 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "reviewer-b", + "reviewer-a" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json new file mode 100644 index 0000000000..a14071c539 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json @@ -0,0 +1,148 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 4, + "min_model_checkers": 3, + "cross_vendor_required": true, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "scanner", + "role": "checker", + "scanner_class": "secret-detection" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author", + "distinct_vendor_from": [ + "reviewer-b", + "reviewer-c" + ], + "min_context_tokens": 200000 + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ], + "distinct_vendor_from": [ + "reviewer-c" + ], + "requires_modality": [ + "text" + ] + }, + { + "name": "reviewer-c", + "role": "cross_vendor_checker", + "distinct_vendor_from": "author", + "distinct_model_from": [ + "reviewer-a", + "reviewer-b" + ], + "requires_feature": [ + "structured-output" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json new file mode 100644 index 0000000000..0827bf013b --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json @@ -0,0 +1,116 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": "author" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json new file mode 100644 index 0000000000..4f9fb9bdd8 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json @@ -0,0 +1,117 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author" + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": "reviewer-a", + "distinct_vendor_from": "reviewer-x" + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json new file mode 100644 index 0000000000..0d5d6c2339 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json @@ -0,0 +1,120 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "verification_topology": { + "C3": { + "min_checkers": 2, + "cross_vendor_required": false, + "roles": [ + { + "name": "author", + "role": "generator" + }, + { + "name": "reviewer-a", + "role": "checker", + "distinct_model_from": "author", + "max_latency_ms": 500 + }, + { + "name": "reviewer-b", + "role": "checker", + "distinct_model_from": [ + "author", + "reviewer-a" + ] + } + ] + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json b/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json index 8a1c195f0b..d67efd8eac 100644 --- a/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json +++ b/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json @@ -53,6 +53,18 @@ "ai-review": { "$ref": "#/$defs/PerClassVerification" } } }, + "verification_topology": { + "description": "Optional per-class verification-topology binding over the verification-topology leaf's vocabulary: WHO verifies, how those verifiers must differ, and the per-class checker floor. An absent key (or an absent class) is NOT a hole: the leaf's shipped floors apply, exactly as escalation_severity falls back to contract defaults. Shipped floor values are FLOORS: tightening is legal, weakening below a shipped value makes the binding invalid — the admission-rule override_justification escape does NOT apply here, the same rule verification_blocking carries. Policy is roles + relational constraints + machine-checkable predicates; every concrete model instance is an org-binding outcome resolved on the binding seam at run time, so no entry here ever names a vendor or instance.", + "type": "object", + "additionalProperties": false, + "properties": { + "C1": { "$ref": "#/$defs/ClassTopology" }, + "C2": { "$ref": "#/$defs/ClassTopology" }, + "C3": { "$ref": "#/$defs/ClassTopology" }, + "C4": { "$ref": "#/$defs/ClassTopology" }, + "C5": { "$ref": "#/$defs/ClassTopology" } + } + }, "promotion_state": { "description": "Keyed by PROMOTABLE cell id only (C2-auto-merge, C3-auto-merge, C3-ai-review-blocking); an entry for a non-promotable cell (C4/C5 merge never promotes) makes the binding invalid. The bound state is a CEILING: consumers must resolve the effective state against live promotion-evidence telemetry before every autonomous dispatch/merge decision, fail-closing to unpromoted when evidence is unavailable.", "type": "object", @@ -226,6 +238,76 @@ } } }, + "ClassTopology": { + "description": "One class's declared verification topology: a list of checker SLOTS plus the floors evaluated over it. All four axes are required — a count with no slot list is a length with no coverage, so a declared class always carries the list its counts are checked against. The checker enforces the floor comparisons, relational-reference resolution, the declared distinctness the leaf requires of every model-adjudicated slot, and the never-vacuous cross_vendor_required rule; this schema fixes the shape.", + "type": "object", + "additionalProperties": false, + "required": ["min_checkers", "min_model_checkers", "cross_vendor_required", "roles"], + "properties": { + "min_checkers": { + "description": "Floor-bounded count of DISTINCT checker slots — a coverage floor, never a list length: deterministic slots are distinguished by scanner class, model-adjudicated slots by declared distinctness, and two slots that resolve identically declare ONE checker.", + "type": "integer", + "minimum": 1 + }, + "min_model_checkers": { + "description": "Floor-bounded count of MODEL-ADJUDICATED checker slots. Exists because a total count cannot express which KIND of coverage is owed: without it a class meets its floor with deterministic slots alone and never faces a model judge.", + "type": "integer", + "minimum": 0 + }, + "cross_vendor_required": { + "description": "Whether the model-adjudicated slots must be pairwise vendor-disjoint AND disjoint from the generator. The knob itself carries that obligation onto every model-adjudicated slot, and it is never vacuously satisfied: asserting it with fewer than two model-adjudicated slots is invalid rather than trivially conforming. Floor-bounded: false where the shipped floor is true is invalid.", + "type": "boolean" + }, + "roles": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/TopologyRole" } + } + } + }, + "TopologyRole": { + "description": "One declared slot in the class's verification pipeline. A slot is filled by either a DETERMINISTIC layer (role checker carrying scanner_class) or a MODEL-ADJUDICATED role; a deterministic layer has no model or vendor identity, so the relational constraints and predicates bind only model-adjudicated slots and are never accepted on a deterministic one. name is the class-local id relational constraints resolve against: a constraint naming a role its own class does not declare is invalid, not ignored — that resolution is what keeps the constraint machine-checkable rather than decorative. A slot name says nothing about what it resolves to, so every model-adjudicated checker slot DECLARES distinct_model_from against the generator and against every other model-adjudicated checker slot; undeclared distinctness is unestablished distinctness. Predicates are requirements a binding can EVALUATE against a candidate instance; a requirement that cannot be evaluated is a preference, and preferences are not policy, so an unrecognized predicate key is rejected rather than carried as decoration.", + "type": "object", + "additionalProperties": false, + "required": ["name", "role"], + "properties": { + "name": { "type": "string", "minLength": 1 }, + "role": { "enum": ["generator", "checker", "cross_vendor_checker", "ranker"] }, + "scanner_class": { + "description": "Marks the slot as a DETERMINISTIC layer and names the scanner class that distinguishes it — two deterministic slots sharing a scanner class resolve identically and declare one checker. Legal on role checker only.", + "type": "string", + "minLength": 1 + }, + "distinct_model_from": { "$ref": "#/$defs/RoleReference" }, + "distinct_vendor_from": { "$ref": "#/$defs/RoleReference" }, + "min_context_tokens": { + "description": "Evaluated against the declared input limit of the bound instance.", + "type": "integer", + "minimum": 1 + }, + "requires_modality": { "$ref": "#/$defs/PredicateTokenList" }, + "requires_feature": { "$ref": "#/$defs/PredicateTokenList" } + } + }, + "RoleReference": { + "description": "Relational-constraint target(s): the class-local name(s) of declared roles this role must differ from. A list is several constraints of one kind; entries must be distinct, because repeating a target restates one constraint while presenting as several.", + "oneOf": [ + { "type": "string", "minLength": 1 }, + { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { "type": "string", "minLength": 1 } + } + ] + }, + "PredicateTokenList": { + "description": "Non-empty, duplicate-free token list. For requires_feature the names are vendor-local, so the binding declares the mapping it resolves them against; a repeated token is one requirement presenting as several.", + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { "type": "string", "minLength": 1 } + }, "Route": { "type": "string", "minLength": 1 }, "EscalationEventClass": { "description": "The escalation event-class registry: the six inherited guardrail classes plus the two additive runner classes. escalation_routes keeps requiring only the six inherited classes; the two runner classes bind additively, and a runner class keyed in escalation_severity must also carry its own escalation_routes entry (the queue destination severity can never supply).", diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json index 8289627570..50cb7cb3b5 100644 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json @@ -830,6 +830,118 @@ "exit": 0, "findings_substrings": [], "credential_roots": "/home/runner" + }, + "topology-all-deterministic-c4.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C4.min_model_checkers: 0 weakens the shipped floor 2", + "verification_topology.C4: cross_vendor_required true with 0 model-adjudicated checker slots" + ], + "credential_roots": "/home/runner" + }, + "topology-checker-count-below-min.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles: declares 2 distinct checker roles against min_checkers 3" + ], + "credential_roots": "/home/runner" + }, + "topology-cross-vendor-decorative.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3: cross_vendor_required true but no declared checker is vendor-distinct from the generator" + ], + "credential_roots": "/home/runner" + }, + "topology-cross-vendor-dropped.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C5.cross_vendor_required: false weakens the shipped floor true — floors may be tightened, never weakened" + ], + "credential_roots": "/home/runner" + }, + "topology-cross-vendor-no-generator.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[\"reviewer-a\"]: role cross_vendor_checker with no generator declared in this class" + ], + "credential_roots": "/home/runner" + }, + "topology-cross-vendor-vacuous.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3: cross_vendor_required true with 1 model-adjudicated checker slot" + ], + "credential_roots": "/home/runner" + }, + "topology-duplicate-modality.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[1].requires_modality: [\"text\",\"text\"] repeats a token" + ], + "credential_roots": "/home/runner" + }, + "topology-duplicate-relation-target.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[2].distinct_model_from: [\"author\",\"author\"] repeats a target" + ], + "credential_roots": "/home/runner" + }, + "topology-duplicate-role-name.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles: \"reviewer-a\" is declared more than once", + "declares 1 distinct checker role against min_checkers 2" + ], + "credential_roots": "/home/runner" + }, + "topology-min-checkers-weakened.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C4.min_checkers: 2 weakens the shipped floor 3 — floors may be tightened, never weakened" + ], + "credential_roots": "/home/runner" + }, + "topology-non-class-key.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology: unknown key \"C6\" (additionalProperties: false)" + ], + "credential_roots": "/home/runner" + }, + "topology-self-reference.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[\"reviewer-b\"].distinct_model_from: names its own role" + ], + "credential_roots": "/home/runner" + }, + "topology-tightened-valid.json": { + "exit": 0, + "findings_substrings": [], + "credential_roots": "/home/runner" + }, + "topology-unconstrained-checker-pair.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles: checker roles \"reviewer-a\" and \"reviewer-b\" carry no distinct_model_from/distinct_vendor_from relation between them" + ], + "credential_roots": "/home/runner" + }, + "topology-undeclared-role.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[\"reviewer-b\"].distinct_vendor_from: \"reviewer-x\" names a role this class does not declare" + ], + "credential_roots": "/home/runner" + }, + "topology-unknown-predicate.json": { + "exit": 1, + "findings_substrings": [ + "verification_topology.C3.roles[1]: unknown key \"max_latency_ms\" (additionalProperties: false)" + ], + "credential_roots": "/home/runner" } }, "quarantined": {}, diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs index 9e4e96f088..4700db806c 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs @@ -6,7 +6,8 @@ // mirrored here, dependency-free) plus the semantic rules the schema cannot // express: matrix merge caps, runtime-marker attestability and pairwise // joint-satisfiability, per-surface class-aware isolation verdicts, promotion -// discipline, and admission floors/precedence. +// discipline, per-class verification-topology floors and coverage, and +// admission floors/precedence. // // Usage: node check-security-binding.mjs [--evidence ] [--probe-evidence-root ] [--egress-hosts ] [--credential-roots ] // Exit 0 = valid (verdicts printed); 1 = findings; 2 = usage/environment error. @@ -365,6 +366,31 @@ const VERIFICATION_FLOORS = { }; const VERIFICATION_STRENGTH = { "not-required": 0, advisory: 1, blocking: 2 }; +// The verification-topology leaf's vocabulary and shipped per-class floors. +// Floors carry the same rule as VERIFICATION_FLOORS: tightening is legal, +// weakening below a shipped value is invalid, and the admission-rule +// override_justification escape applies to admission rules ONLY, never here. +// An ABSENT verification_topology key (or an absent class) is not a hole: +// the leaf's shipped floors apply, exactly as escalation_severity falls back +// to contract defaults. +const TOPOLOGY_ROLES = ["generator", "checker", "cross_vendor_checker", "ranker"]; +// The roles that count toward min_checkers: a cross_vendor_checker IS a +// checker, additionally constrained to a different vendor from the generator. +const CHECKER_TYPED_ROLES = new Set(["checker", "cross_vendor_checker"]); +const RELATIONAL_CONSTRAINTS = ["distinct_model_from", "distinct_vendor_from"]; +// min_model_checkers exists because a total count cannot express which KIND +// of coverage is owed: without it a class meets min_checkers with +// deterministic slots alone, never faces a model judge, and +// cross_vendor_required then binds an empty set of model slots and is +// satisfied by declaring nothing. +const TOPOLOGY_FLOORS = { + C1: { min_checkers: 1, min_model_checkers: 0, cross_vendor_required: false }, + C2: { min_checkers: 1, min_model_checkers: 0, cross_vendor_required: false }, + C3: { min_checkers: 2, min_model_checkers: 1, cross_vendor_required: false }, + C4: { min_checkers: 3, min_model_checkers: 2, cross_vendor_required: true }, + C5: { min_checkers: 3, min_model_checkers: 2, cross_vendor_required: true }, +}; + const PROMOTABLE_CELLS = new Set(["C2-auto-merge", "C3-auto-merge", "C3-ai-review-blocking"]); // C3 auto-merge's evidence predicate builds on the C2 auto-merge track record @@ -469,6 +495,7 @@ function validateStructure(binding) { "isolation_bindings", "merge_policy", "verification_blocking", + "verification_topology", "promotion_state", "escalation_routes", "escalation_severity", @@ -605,6 +632,20 @@ function validateStructure(binding) { } } + if (Object.hasOwn(binding, "verification_topology")) { + if (!isPlainObject(binding.verification_topology)) { + findings.push("verification_topology: must be an object keyed by work class"); + } else { + checkAllowedKeys(binding.verification_topology, WORK_CLASSES, "verification_topology"); + for (const workClass of WORK_CLASSES) { + // An absent class is not a hole: the verification-topology leaf's + // shipped floors apply to it, so only declared classes validate. + if (!Object.hasOwn(binding.verification_topology, workClass)) continue; + validateClassTopologyStructure(binding.verification_topology[workClass], `verification_topology.${workClass}`); + } + } + } + if (Object.hasOwn(binding, "promotion_state")) { if (!isPlainObject(binding.promotion_state)) { findings.push("promotion_state: must be an object keyed by promotable cell id"); @@ -901,6 +942,90 @@ function validateTemporalClassificationHome(ruleHome, where) { } } +// One class's declared verification topology (mirrors $defs/ClassTopology). +// All three axes are required on a declared class: a count with no role list +// is a length with no coverage, so the roles the count is evaluated against +// always travel with it. Predicate keys are a CLOSED set — a predicate is a +// requirement a binding can EVALUATE against a candidate instance, and an +// unrecognized token cannot be evaluated, so it is rejected rather than +// carried as decoration. +function validateClassTopologyStructure(topology, where) { + if (!isPlainObject(topology)) { + findings.push(`${where}: must be an object binding the class's declared topology ({min_checkers, min_model_checkers, cross_vendor_required, roles})`); + return; + } + const topologyKeys = ["min_checkers", "min_model_checkers", "cross_vendor_required", "roles"]; + checkAllowedKeys(topology, topologyKeys, where); + for (const key of topologyKeys) { + if (!Object.hasOwn(topology, key)) { + findings.push(`${where}.${key}: required key missing — a declared class topology carries all four axes, because a count without its role list (or a role list without its count) cannot be evaluated`); + } + } + if (Object.hasOwn(topology, "min_checkers") && (!Number.isInteger(topology.min_checkers) || topology.min_checkers < 1)) { + findings.push(`${where}.min_checkers: must be an integer >= 1`); + } + if (Object.hasOwn(topology, "min_model_checkers") && (!Number.isInteger(topology.min_model_checkers) || topology.min_model_checkers < 0)) { + findings.push(`${where}.min_model_checkers: must be an integer >= 0`); + } + if (Object.hasOwn(topology, "cross_vendor_required") && typeof topology.cross_vendor_required !== "boolean") { + findings.push(`${where}.cross_vendor_required: must be a boolean`); + } + if (!Object.hasOwn(topology, "roles")) return; + if (!Array.isArray(topology.roles) || topology.roles.length === 0) { + findings.push(`${where}.roles: must be a non-empty array of declared role entries`); + return; + } + topology.roles.forEach((entry, index) => { + const entryWhere = `${where}.roles[${index}]`; + if (!isPlainObject(entry)) { + findings.push(`${entryWhere}: must be an object`); + return; + } + checkAllowedKeys( + entry, + ["name", "role", "scanner_class", "distinct_model_from", "distinct_vendor_from", "min_context_tokens", "requires_modality", "requires_feature"], + entryWhere, + ); + if (Object.hasOwn(entry, "scanner_class") && !isNonEmptyString(entry.scanner_class)) { + findings.push(`${entryWhere}.scanner_class: must be a non-empty string naming the scanner class that distinguishes this deterministic slot`); + } + if (!isNonEmptyString(entry.name)) { + findings.push(`${entryWhere}.name: missing or empty — the class-local id relational constraints resolve against`); + } + checkEnum(entry.role, TOPOLOGY_ROLES, `${entryWhere}.role`); + for (const constraint of RELATIONAL_CONSTRAINTS) { + if (!Object.hasOwn(entry, constraint)) continue; + const value = entry[constraint]; + const targets = typeof value === "string" ? [value] : Array.isArray(value) ? value : null; + if (targets === null || targets.length === 0 || targets.some((target) => !isNonEmptyString(target))) { + findings.push(`${entryWhere}.${constraint}: must be a non-empty role name or a non-empty array of role names`); + continue; + } + if (new Set(targets).size !== targets.length) { + findings.push( + `${entryWhere}.${constraint}: ${JSON.stringify(value)} repeats a target — listing one role twice restates a single constraint while presenting as several`, + ); + } + } + if (Object.hasOwn(entry, "min_context_tokens") && (!Number.isInteger(entry.min_context_tokens) || entry.min_context_tokens < 1)) { + findings.push(`${entryWhere}.min_context_tokens: must be an integer >= 1 — the predicate evaluates against the declared input limit of the bound instance`); + } + for (const predicate of ["requires_modality", "requires_feature"]) { + if (!Object.hasOwn(entry, predicate)) continue; + const value = entry[predicate]; + if (!Array.isArray(value) || value.length === 0 || value.some((token) => !isNonEmptyString(token))) { + findings.push(`${entryWhere}.${predicate}: must be a non-empty array of non-empty string tokens`); + continue; + } + if (new Set(value).size !== value.length) { + findings.push( + `${entryWhere}.${predicate}: ${JSON.stringify(value)} repeats a token — a repeated requirement is one requirement presenting as several`, + ); + } + } + }); +} + // --- Semantic rules the schema cannot express --- function levelNumber(token) { @@ -1775,6 +1900,23 @@ function checkSemantics(binding, probeRoot, egressAllowList, credentialRoots) { } } + // Verification-topology floors and coverage per declared class. An absent + // verification_topology (or an absent class within it) is NOT a hole: the + // verification-topology leaf's shipped floors apply, exactly as + // escalation_severity falls back to contract defaults — so only declared + // classes are checked, and each on all three axes: the floors + // (tighten-only, no override escape — the same rule verification_blocking + // carries), the reference resolution that keeps relational constraints + // machine-checkable, and the distinctness coverage that keeps min_checkers + // a count of distinct checkers rather than a length. + if (isPlainObject(binding.verification_topology)) { + for (const workClass of WORK_CLASSES) { + const topology = binding.verification_topology[workClass]; + if (!isPlainObject(topology)) continue; + checkTopologySemantics(topology, workClass); + } + } + // No notification-routability rule exists for escalation_severity on // purpose: severity selects only the NOTIFICATION fan-out layered on the // filed item — it never redirects the item, whose queue destination stays @@ -1845,6 +1987,226 @@ function checkAdmissionSemantics(rules) { } } +// A relational constraint's target list, tolerant of structurally invalid +// values (those already produced their structural finding): a string is one +// target, an array is several. +function topologyConstraintTargets(entry, constraint) { + const value = entry[constraint]; + if (typeof value === "string") return isNonEmptyString(value) ? [value] : []; + if (Array.isArray(value)) return value.filter(isNonEmptyString); + return []; +} + +function checkTopologySemantics(topology, workClass) { + const where = `verification_topology.${workClass}`; + const floor = TOPOLOGY_FLOORS[workClass]; + if (Number.isInteger(topology.min_checkers) && topology.min_checkers < floor.min_checkers) { + findings.push( + `${where}.min_checkers: ${topology.min_checkers} weakens the shipped floor ${floor.min_checkers} — floors may be tightened, never weakened (override_justification applies to admission rules only)`, + ); + } + if ( + Number.isInteger(topology.min_model_checkers) && + topology.min_model_checkers < floor.min_model_checkers + ) { + findings.push( + `${where}.min_model_checkers: ${topology.min_model_checkers} weakens the shipped floor ${floor.min_model_checkers} — floors may be tightened, never weakened (override_justification applies to admission rules only)`, + ); + } + if (topology.cross_vendor_required === false && floor.cross_vendor_required) { + findings.push( + `${where}.cross_vendor_required: false weakens the shipped floor true — floors may be tightened, never weakened (override_justification applies to admission rules only)`, + ); + } + if (!Array.isArray(topology.roles)) return; + const entries = topology.roles.filter((entry) => isPlainObject(entry) && isNonEmptyString(entry.name)); + // Distinct names, not entry count: two entries sharing one name declare ONE + // role while presenting as two, so the duplicate is rejected and only the + // first declaration seats the name. + const declared = new Map(); + for (const entry of entries) { + if (declared.has(entry.name)) { + findings.push( + `${where}.roles: ${JSON.stringify(entry.name)} is declared more than once — role names are the class-local identity relational constraints resolve against, so a repeated name declares one role while presenting as several`, + ); + } else { + declared.set(entry.name, entry); + } + } + // Reference resolution: a relational constraint naming a role its own class + // does not declare is INVALID, not ignored — this resolution is what makes + // the constraint machine-checkable rather than decorative. A self-reference + // is equally unevaluable: a role cannot be distinct from itself. + for (const entry of entries) { + for (const constraint of RELATIONAL_CONSTRAINTS) { + if (!Object.hasOwn(entry, constraint)) continue; + for (const target of topologyConstraintTargets(entry, constraint)) { + if (target === entry.name) { + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}].${constraint}: names its own role — a role cannot be distinct from itself, so a self-reference can never be evaluated`, + ); + } else if (!declared.has(target)) { + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}].${constraint}: ${JSON.stringify(target)} names a role this class does not declare — a relational constraint resolves against its own class's role list, so an undeclared reference can never be evaluated; invalid, not ignored`, + ); + } + } + } + } + // A cross_vendor_checker's DEFINING constraint references the generator — + // "a checker additionally constrained to a different vendor from the + // generator" — so declaring one in a class with no declared generator role + // leaves that constraint without a referent: unevaluable, under the same + // invalid-not-ignored rule as any other unresolvable reference. + const generatorNames = new Set( + [...declared.values()].filter((entry) => entry.role === "generator").map((entry) => entry.name), + ); + if (generatorNames.size === 0) { + for (const entry of declared.values()) { + if (entry.role !== "cross_vendor_checker") continue; + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}]: role cross_vendor_checker with no generator declared in this class — the role is a checker constrained to a different vendor from the GENERATOR, so without a declared generator its defining constraint has no referent and can never be evaluated; declare the generator role, or use role checker`, + ); + } + } + const checkers = [...declared.values()].filter((entry) => CHECKER_TYPED_ROLES.has(entry.role)); + // A slot is filled by either a DETERMINISTIC layer or a MODEL-ADJUDICATED + // role. A deterministic layer has no model or vendor identity, so the + // relational constraints and predicates cannot be evaluated against one and + // are rejected on it rather than silently ignored. + const deterministic = checkers.filter((entry) => isNonEmptyString(entry.scanner_class)); + const modelCheckers = checkers.filter((entry) => !isNonEmptyString(entry.scanner_class)); + for (const entry of deterministic) { + if (entry.role !== "checker") { + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}]: scanner_class on role ${entry.role} — scanner_class marks a DETERMINISTIC slot, which has no vendor identity, so it is legal on role checker only`, + ); + } + for (const key of [...RELATIONAL_CONSTRAINTS, "min_context_tokens", "requires_modality", "requires_feature"]) { + if (!Object.hasOwn(entry, key)) continue; + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}].${key}: declared on a deterministic slot — a deterministic layer has no model or vendor identity, so this constraint can never be evaluated against it; drop it, or drop scanner_class to declare a model-adjudicated slot`, + ); + } + } + // Deterministic slots are distinguished by SCANNER CLASS: two sharing one + // scanner class resolve identically and declare one checker. + const scannerClasses = new Map(); + for (const entry of deterministic) { + const seen = scannerClasses.get(entry.scanner_class); + if (seen !== undefined) { + findings.push( + `${where}.roles: deterministic slots ${JSON.stringify(seen)} and ${JSON.stringify(entry.name)} share scanner_class ${JSON.stringify(entry.scanner_class)} — two slots that resolve identically declare ONE checker, so the pair counts once toward min_checkers`, + ); + } else { + scannerClasses.set(entry.scanner_class, entry.name); + } + } + // A slot NAME says nothing about what it resolves to, so distinctness that + // is only intended is not distinctness: every model-adjudicated checker + // DECLARES distinct_model_from against the generator. Undeclared is the + // unevaluable case, which is the same failure as declaring none. + for (const entry of modelCheckers) { + if (generatorNames.size === 0) continue; + const targets = RELATIONAL_CONSTRAINTS.flatMap((constraint) => topologyConstraintTargets(entry, constraint)); + if (targets.some((target) => generatorNames.has(target))) continue; + findings.push( + `${where}.roles[${JSON.stringify(entry.name)}]: model-adjudicated checker with no distinctness relation to the generator — a model judging its own output measures its own preference rather than the artifact, and an undeclared constraint is unestablished, not generous; declare distinct_model_from against the generator role`, + ); + } + const minModelCheckers = Number.isInteger(topology.min_model_checkers) + ? topology.min_model_checkers + : floor.min_model_checkers; + if (modelCheckers.length < minModelCheckers) { + findings.push( + `${where}.roles: declares ${modelCheckers.length} model-adjudicated checker slot${modelCheckers.length === 1 ? "" : "s"} against min_model_checkers ${minModelCheckers} — a total count cannot express which KIND of coverage is owed, so a class meeting min_checkers with deterministic slots alone never faces the model judge its verification cell requires`, + ); + } + // Coverage, not length: min_checkers is evaluated over the DISTINCT + // checker-typed roles the class declares — a list that cannot seat the + // count makes the declared topology unsatisfiable. + const minCheckers = Number.isInteger(topology.min_checkers) ? topology.min_checkers : floor.min_checkers; + if (checkers.length < minCheckers) { + findings.push( + `${where}.roles: declares ${checkers.length} distinct checker role${checkers.length === 1 ? "" : "s"} against min_checkers ${minCheckers} — min_checkers counts DISTINCT checker roles drawn from the declared role list, so a list that cannot seat the count makes the topology unsatisfiable`, + ); + } + // Count is not coverage: two declared checker roles with no distinctness + // relation between them can both resolve to ONE instance at run time, and + // N runs of a single instance count as one checker — they share the + // failure the count exists to catch. Every pair of counted checkers must + // be held distinct by a relational constraint in at least one direction + // (distinct_vendor_from implies distinct model: disjoint vendors never + // share an instance). + const holdsDistinct = (a, b) => + RELATIONAL_CONSTRAINTS.some( + (constraint) => + topologyConstraintTargets(a, constraint).includes(b.name) || + topologyConstraintTargets(b, constraint).includes(a.name), + ); + // Scoped to the MODEL-adjudicated slots. A deterministic slot cannot carry a + // relational constraint at all — it has no model or vendor identity — and it + // can never resolve to the same thing as a model judge, so a cross-kind pair + // is distinct by construction. Deterministic slots are held apart from each + // other by scanner class above. + for (let i = 0; i < modelCheckers.length; i += 1) { + for (let j = i + 1; j < modelCheckers.length; j += 1) { + if (!holdsDistinct(modelCheckers[i], modelCheckers[j])) { + findings.push( + `${where}.roles: checker roles ${JSON.stringify(modelCheckers[i].name)} and ${JSON.stringify(modelCheckers[j].name)} carry no distinct_model_from/distinct_vendor_from relation between them — nothing keeps both from resolving to one instance, and N runs of a single instance count as ONE checker toward min_checkers; constrain the pair distinct in at least one direction`, + ); + } + } + } + // cross_vendor_required is enforceable only where the role list can seat + // it: a cross_vendor_checker is definitionally vendor-distinct from the + // generator, and a plain checker qualifies when constrained + // distinct_vendor_from a declared generator role. A true knob with neither + // is a requirement the declared topology can never evaluate — decorative, + // not policy. + if (topology.cross_vendor_required === true) { + // Never vacuously satisfied. The knob's whole reason is that same-vendor + // checkers share failure modes, so agreement between them is weaker + // evidence than its count suggests — a class asserting it over fewer than + // two model-adjudicated slots satisfies neither the constraint nor the + // reason for it, and one asserting it over ZERO binds the empty set and is + // satisfied by declaring nothing. + if (modelCheckers.length < 2) { + findings.push( + `${where}: cross_vendor_required true with ${modelCheckers.length} model-adjudicated checker slot${modelCheckers.length === 1 ? "" : "s"} — the requirement is vendor disjointness AMONG the model slots, so fewer than two makes it vacuous rather than trivially conforming; declare at least two model-adjudicated checkers or bind cross_vendor_required false where the floor allows`, + ); + } + const seated = checkers.some( + (entry) => + entry.role === "cross_vendor_checker" || + topologyConstraintTargets(entry, "distinct_vendor_from").some((target) => generatorNames.has(target)), + ); + if (!seated) { + findings.push( + `${where}: cross_vendor_required true but no declared checker is vendor-distinct from the generator — declare a cross_vendor_checker role, or constrain a checker distinct_vendor_from a declared generator role; a requirement the role list cannot seat is decorative, not policy`, + ); + } + // Disjointness among the model slots, not merely against the generator: + // {V2, V2, V2} against a V1 generator satisfies a generator-only reading + // while every checker shares a vendor with every other — precisely the + // shared-failure-mode case the knob exists to break up. + for (let i = 0; i < modelCheckers.length; i += 1) { + for (let j = i + 1; j < modelCheckers.length; j += 1) { + const a = modelCheckers[i]; + const b = modelCheckers[j]; + const disjoint = + topologyConstraintTargets(a, "distinct_vendor_from").includes(b.name) || + topologyConstraintTargets(b, "distinct_vendor_from").includes(a.name); + if (!disjoint) { + findings.push( + `${where}.roles: model-adjudicated checkers ${JSON.stringify(a.name)} and ${JSON.stringify(b.name)} carry no distinct_vendor_from relation between them while cross_vendor_required is true — vendor disjointness is required AMONG the model slots, not only against the generator, because same-vendor checkers share failure modes and their agreement is weaker evidence than its count suggests`, + ); + } + } + } + } +} + // --- Evaluation mode: effective promotion state from an evidence source --- function resolveEffectivePromotion(binding, evidencePath) { From 5b0ca1deb0e0ba49ad8cd2eb0c7446c3cf2cc3ef Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 22:00:08 -0400 Subject: [PATCH 13/17] docs(topics): record Phases 2 and 3 shipped, and the defect they share MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both phases landed, and both were repaired under independent audit for the same reason the phase before them was: a count that does not guarantee the coverage it exists for. Three occurrences across two phases stops being an incident and becomes this effort's signature failure, so it is recorded as a generalization rather than as three separate corrections — whenever a rule counts things, ask what it would accept if every counted thing were identical. Phase 3 also surfaced something no review round found: the pairwise-distinctness rule and the deterministic-slot rule were each sound and jointly unsatisfiable, making a conforming binding unrepresentable. Only executing them showed it, which is worth carrying into the phases that still have checks to write. Q21, Q22 and Q23 are marked resolved where the document still called them USER-RESERVED, and Q22's trigger wording is ratified as drafted. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 52 ++++++++++++++++++-- 1 file changed, 47 insertions(+), 5 deletions(-) diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md index b331277996..3a7990e385 100644 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ b/docs/topics/docker-sandbox-substrate/PLAN.md @@ -345,7 +345,28 @@ Criterion 3. Delivers the Q20 resolution above. - A re-probe transcript exists under `.work/docker-sandbox-substrate/` recording the new `transport_outcome` and `workspace_host_write_contained` blocks from a live run. -#### Phase 2: Verification-topology contract leaf + matrix column [TODO] +#### Phase 2: Verification-topology contract leaf + matrix column [DONE] + +**SHIPPED** as `f7d96afc`, then repaired twice under independent audit (`0c75e0a9`, `8172ab67`). + +**Both repair rounds found the SAME defect shape the phase before it did — a count that does not +guarantee the coverage it exists for.** Round 1: `min_checkers` counted role TYPES, so a class +declaring `[A, A, B]` satisfied a floor of 3. Round 2, inside the repair itself: relations and +predicates bind only model-adjudicated slots, so three DETERMINISTIC slots met `C4`'s floor while +`cross_vendor_required` bound the empty set of model slots and was vacuously satisfied — a binding +valid with no model judge at all, against a matrix cell that mandates AI review. **Three occurrences +across two phases makes this the effort's signature failure, not an incident.** The generalization +worth carrying: whenever a rule counts things, ask what it would accept if every counted thing were +identical. + +Other findings the audit closed: distinctness stated as "implied", which a validator cannot act on +because a slot NAME says nothing about what it resolves to; a budget ceiling written as invalidating, +which would have made this leaf the single enforcing exception to the matrix's own out-of-scope +statement on cost; three pointers citing support no file carried; and a deliberation clause resting +on an uncited measurement — replaced by the argument that is actually analytic, with the measurement +and its confidence grade moved to issue #2110. + +#### Phase 2 (original brief) Review: architecture @@ -379,7 +400,24 @@ Criterion 4, and the vocabulary Phases 3–5 cite. Documentation only — no sch the glance-layer table contains a row pointing at `guardrails/verification-topology.md`. - Every plugin-internal link in the new leaf resolves (`skill-reference-verify` hook passes). -#### Phase 3: Per-class verification floors on the security binding [TODO] +#### Phase 3: Per-class verification floors on the security binding [DONE] + +**SHIPPED** as `16a50974`. 522 checks / 148 fixtures, manifest diff purely additive (16 added, 0 +removed, 0 pre-existing `findings_substrings` modified — verified semantically, not by eye). + +**The floor table gained a fourth axis the brief did not name.** `min_model_checkers` +(`C1` 0 · `C2` 0 · `C3` 1 · `C4` 2 · `C5` 2) exists because a total count cannot express which KIND +of coverage is owed. Without it the `C4` all-deterministic binding above is valid. +`cross_vendor_required` is now never vacuously satisfiable, and vendor disjointness holds among the +model slots rather than only against the generator. + +**One contradiction surfaced only by running it:** the pairwise-distinctness check demanded a +relational constraint between every checker pair, while the deterministic/model split rejects those +same constraints on deterministic slots — so a conforming binding was unrepresentable. Scoped to +model slots, since a cross-kind pair is distinct by construction. **A rule pair can be individually +sound and jointly unsatisfiable; only executing it shows that.** + +#### Phase 3 (original brief) Review: security @@ -648,6 +686,10 @@ explicitly not assumed to have fired.** Wording is for review. Each is recorded as DEFERRED WITH A TRIGGER, never as rejected — the ladder's own "Rejected axis" section is reserved for what was deliberately not chosen, which these are not. +**Wording RATIFIED 2026-08-11 as drafted.** The three triggers stand verbatim; the review the user +reserved is closed. Nothing downstream depends on the phrasing, so a later revision costs a wording +commit and no rework. + #### Q23 — event-triggered re-verification, with a staleness bound as backstop **Decision: re-verify on events that could change the probed property; cap evidence age separately. @@ -770,9 +812,9 @@ slice that would delegate cleanly. ### Open questions -- OPEN DECISION 1 (floor values) must resolve before Phase 3 starts. Nothing else is blocked. -- Q21, Q22, Q23 remain USER-RESERVED. No phase depends on any of them, and Phase 1's multi-target - egress leg deliberately strengthens the probe without deciding Q21. +- OPEN DECISION 1 (floor values) resolved 2026-08-11; Phase 3 shipped against it. +- Q21, Q22, Q23 were USER-RESERVED and are RESOLVED 2026-08-11 — see the resolutions below. Earlier + USER-RESERVED markers in this document predate that and are stale where they conflict. ### Handoff to implementation From 6c3382ae104084fc829f2c2323512dfcd475f538 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 22:56:31 -0400 Subject: [PATCH 14/17] feat(autonomy): require the probe to cover every ratified component-reachable host MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Phase 1 hardened the egress assertion but left target selection unconstrained, and target selection is where the assertion is won or lost. A probe sampling only hosts the installed components do not reach certifies a boundary that is in fact open — measured, not theorized: 201,961 bytes of origin data crossed a global default-deny through a component-installed allow rule, with the origin's own CA on the wire. The ladder gains the property as a DIRECTION rather than a rule shaped around one vendor's component system: an L2+ surface's policy layer must let installed components subtract reachability and never add it. That generalizes to extension permissions, admission control, and any engine composing rules from parts, and it survives the classes-never-vendors rule the alternative would have broken. The ratified host set rides the level binding, not the transcript and not a CLI argument. A first design used a CLI seam; running it showed its no-hosts sentinel binding L2 with zero evidence, and the seam put the fact the check turns on where no reviewer of the binding or the capture could see it. substrate_class already set the precedent in this same function, and for the same reason: a transcript value is capture evidence the executing agent could doctor. Coverage is FULL, not at-least-one. Each ratified destination is a separate policy decision, so covering one says nothing about the rest — the same count-versus-coverage distinction three findings in this effort have now turned on. A partial-coverage fixture pins it, because an at-least-one implementation passes every other test in the set. The empty list is the explicit claim that a surface installs nothing carrying policy rules of its own; an absent field leaves the level unproven. Both paths are fixtured, since the empty list is what unlocks a level and was otherwise untested. The new leg runs LAST in verifyProbeTranscript, which returns the first problem it finds while every probe-evidence fixture pins its own rejection reason. An earlier placement would answer for all of them: the suite stays green, the names keep promising one thing, and what they prove has quietly become something else. Two residues are recorded rather than implied. No capture can prove the ratified set is complete — a component requesting an unratified destination is invisible to every transcript, and that is where the human ratifying the binding takes responsibility. Nothing at all proves the probe ran with those components installed. Also drops the last vendor component-system name from the contract surface, and corrects the setup skill's probe count, which still said two after the third assertion shipped. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- .../reference/guardrails/isolation-ladder.md | 13 ++- plugins/autonomy/skills/setup/SKILL.md | 18 ++-- .../admission-weakened-no-justification.json | 3 + .../ambiguous-runtime-markers.json | 6 ++ .../security-binding/as112-v6-valid.json | 38 ++++++- .../security-binding/c2-auto-unratified.json | 3 + .../security-binding/c3-auto-unratified.json | 3 + .../c3-blocking-unratified.json | 3 + .../security-binding/c4-auto-merge.json | 3 + .../component-reach-empty-valid.json | 95 ++++++++++++++++++ .../equal-specificity-tie.json | 3 + .../escalation-ack-invalid.json | 38 ++++++- .../security-binding/escalation-ack-loop.json | 38 ++++++- .../escalation-push-without-channel.json | 42 ++++++-- .../escalation-runner-route-missing.json | 38 ++++++- .../escalation-severity-bad-token.json | 38 ++++++- ...scalation-severity-tracker-only-valid.json | 38 ++++++- .../escalation-severity-valid.json | 47 +++++++-- .../hosted-ephemeral-l3-valid.json | 38 ++++++- .../l3-ratified-class-container.json | 38 ++++++- .../marker-prototype-key.json | 41 +++++++- .../missing-probe-evidence.json | 3 + .../missing-substrate-class.json | 40 ++++++-- .../multi-host-egress-valid.json | 38 ++++++- .../probe-evidence-6to4-relay.json | 38 ++++++- .../probe-evidence-6to4-v6.json | 38 ++++++- .../probe-evidence-bare-metadata.json | 38 ++++++- .../probe-evidence-benchmarking-v6.json | 38 ++++++- .../probe-evidence-bogus-metadata.json | 38 ++++++- ...robe-evidence-component-reach-partial.json | 98 +++++++++++++++++++ ...be-evidence-component-reach-uncovered.json | 98 +++++++++++++++++++ ...e-evidence-component-reach-unratified.json | 94 ++++++++++++++++++ .../probe-evidence-cred-outer-missing.json | 38 ++++++- .../probe-evidence-cred-outer-nonzero.json | 38 ++++++- .../probe-evidence-credpath-code-count.json | 38 ++++++- .../probe-evidence-credpath-partial-zero.json | 38 ++++++- ...robe-evidence-credroots-case-mismatch.json | 38 ++++++- .../probe-evidence-credroots-drive-root.json | 38 ++++++- ...robe-evidence-credroots-relative-root.json | 38 ++++++- .../probe-evidence-credroots-root-slash.json | 38 ++++++- ...probe-evidence-credroots-unconfigured.json | 38 ++++++- .../probe-evidence-discard-only.json | 38 ++++++- .../probe-evidence-doc-3fff.json | 38 ++++++- .../probe-evidence-docnet-host.json | 38 ++++++- .../probe-evidence-dot-traversal.json | 38 ++++++- .../probe-evidence-dummy-prefix.json | 38 ++++++- ...robe-evidence-egress-client-not-ready.json | 3 + .../probe-evidence-egress-duplicate-host.json | 3 + ...vidence-egress-fingerprint-on-nonpeer.json | 3 + .../probe-evidence-egress-no-transport.json | 3 + .../probe-evidence-egress-peer-match.json | 3 + .../probe-evidence-egress-single-target.json | 3 + ...probe-evidence-egress-transport-count.json | 3 + ...probe-evidence-egress-transport-token.json | 3 + ...nce-egress-zero-exit-peer-substituted.json | 3 + .../probe-evidence-encoded-loopback.json | 38 ++++++- .../probe-evidence-expanded-loopback.json | 38 ++++++- .../probe-evidence-expansion-ephemeral.json | 38 ++++++- ...robe-evidence-expansion-tail-mismatch.json | 38 ++++++- .../probe-evidence-generic-env-token.json | 38 ++++++- .../probe-evidence-host-home-mount.json | 38 ++++++- .../probe-evidence-imds-http-error.json | 38 ++++++- .../probe-evidence-invalid-tld.json | 38 ++++++- .../probe-evidence-invented-env-token.json | 38 ++++++- .../probe-evidence-invented-home-user.json | 38 ++++++- .../probe-evidence-ipv4-compat.json | 38 ++++++- .../probe-evidence-l3-container.json | 38 ++++++- .../probe-evidence-level-mismatch.json | 3 + .../probe-evidence-local-host.json | 38 ++++++- .../probe-evidence-malformed-dns.json | 38 ++++++- ...-evidence-metadata-expansion-mismatch.json | 38 ++++++- .../probe-evidence-metadata-port.json | 38 ++++++- ...robe-evidence-metadata-route-boundary.json | 38 ++++++- .../probe-evidence-metadata-route.json | 38 ++++++- .../probe-evidence-metadata-scheme.json | 38 ++++++- .../probe-evidence-missing-host-expanded.json | 38 ++++++- .../probe-evidence-missing-outer-exit.json | 38 ++++++- .../probe-evidence-missing-probed-at.json | 38 ++++++- .../probe-evidence-missing-target.json | 3 + .../probe-evidence-missing-transport.json | 38 ++++++- .../probe-evidence-multi-host-private.json | 38 ++++++- .../probe-evidence-nat64-local.json | 38 ++++++- .../probe-evidence-nat64-private-embed.json | 38 ++++++- .../probe-evidence-outer-egress-failed.json | 38 ++++++- .../probe-evidence-protocol-assignment.json | 38 ++++++- ...robe-evidence-ratified-class-mismatch.json | 38 ++++++- .../probe-evidence-root-escape.json | 38 ++++++- .../probe-evidence-site-local-v6.json | 38 ++++++- .../probe-evidence-srv6-sid.json | 38 ++++++- .../probe-evidence-teredo.json | 38 ++++++- .../probe-evidence-transport-cross-type.json | 38 ++++++- .../probe-evidence-unc-credpath.json | 38 ++++++- .../probe-evidence-uri-host.json | 38 ++++++- ...probe-evidence-workspace-canary-count.json | 3 + ...e-evidence-workspace-canary-duplicate.json | 3 + ...robe-evidence-workspace-canary-shapes.json | 3 + ...e-evidence-workspace-canary-traversal.json | 3 + ...obe-evidence-workspace-canary-visible.json | 3 + ...evidence-workspace-git-digest-changed.json | 3 + ...e-evidence-workspace-inner-code-empty.json | 3 + .../probe-evidence-workspace-missing.json | 3 + .../probe-evidence-workspace-no-teardown.json | 3 + ...obe-evidence-workspace-not-applicable.json | 3 + .../probe-evidence-zero-exit.json | 3 + .../security-binding/promoted-c2.json | 3 + .../security-binding/promoted-c2c3-late.json | 3 + .../security-binding/promoted-c2c3.json | 3 + .../promoted-c2reearned-c3.json | 3 + .../security-binding/promoted-c3-only.json | 3 + .../promotion-nonpromotable.json | 3 + .../ratified-at-calendar-invalid.json | 38 ++++++- .../security-binding/ratified-at-non-iso.json | 3 + .../ratified-at-offset-invalid.json | 38 ++++++- .../ratified-at-submilli.json | 38 ++++++- .../same-surface-level-ambiguity.json | 6 ++ .../security-binding/ssh-id-rsa-valid.json | 38 ++++++- .../temporal-classification-bare-string.json | 38 ++++++- .../temporal-classification-valid.json | 38 ++++++- .../temporal-duplicate-producer-identity.json | 38 ++++++- .../temporal-empty-source-surface.json | 38 ++++++- .../temporal-identity-grammar.json | 38 ++++++- .../temporal-missing-producer-identity.json | 38 ++++++- .../temporal-missing-run-link-prefix.json | 38 ++++++- ...temporal-shared-run-link-prefix-valid.json | 38 ++++++- .../temporal-shared-source-surface.json | 38 ++++++- .../topology-all-deterministic-c4.json | 3 + .../topology-checker-count-below-min.json | 3 + .../topology-cross-vendor-decorative.json | 3 + .../topology-cross-vendor-dropped.json | 3 + .../topology-cross-vendor-no-generator.json | 3 + .../topology-cross-vendor-vacuous.json | 3 + .../topology-duplicate-modality.json | 3 + .../topology-duplicate-relation-target.json | 3 + .../topology-duplicate-role-name.json | 3 + .../topology-min-checkers-weakened.json | 3 + .../topology-non-class-key.json | 3 + .../topology-self-reference.json | 3 + .../topology-tightened-valid.json | 3 + .../topology-unconstrained-checker-pair.json | 3 + .../topology-undeclared-role.json | 3 + .../topology-unknown-predicate.json | 3 + .../two-level-surface-valid.json | 6 ++ .../unverifiable-probe-evidence.json | 3 + .../fixtures/security-binding/valid.json | 38 ++++++- .../security-binding/vendor-hosted-auto.json | 3 + .../verification-below-floor.json | 3 + .../guardrails-security-binding.schema.json | 7 +- ...curity-binding.fixtures.test.manifest.json | 26 +++++ .../setup/scripts/check-security-binding.mjs | 41 +++++++- .../skills/setup/templates/isolation-probe.md | 30 +++++- 150 files changed, 3408 insertions(+), 429 deletions(-) create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/component-reach-empty-valid.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-partial.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-uncovered.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-unratified.json diff --git a/plugins/autonomy/reference/guardrails/isolation-ladder.md b/plugins/autonomy/reference/guardrails/isolation-ladder.md index c69bf642f7..30cc968c3a 100644 --- a/plugins/autonomy/reference/guardrails/isolation-ladder.md +++ b/plugins/autonomy/reference/guardrails/isolation-ladder.md @@ -13,7 +13,8 @@ document names substrate CLASSES as marked examples only, never an instance list tools, hooks, and protocol-connected tool surfaces still execute on the host. An attended ergonomics tier — NOT an autonomy tier. - **`L2` — whole-process OS-enforced boundary with default-deny egress, credential - protection, and contained workspace host-writes.** The MINIMUM for any unattended run. + protection, contained workspace host-writes, and policy nothing the run can install may + widen.** The MINIMUM for any unattended run. Free-path substrate classes (marked examples, not an instance list): a whole-process OS-sandbox wrap; a container with a default-deny egress firewall. @@ -23,6 +24,16 @@ document names substrate CLASSES as marked examples only, never an instance list host later executes — a version-control config file is a command key ring, and one of its keys fires on a read-only-looking status call. Scope is deliberately WRITE containment: read exposure is not covered at this level, and a copy-on-read workspace leaves reads fully open. + + The widening clause names a DIRECTION, because that is what generalizes. An additive policy + layer — one whose installed components carry rules of their own, in the sense of extension + permissions, admission control, or any engine that composes its rules from parts — meets `L2` + only where a component's rules can subtract reachability and never add it. A surface where an + installed component can add an allow rule the base policy denies is not `L2` however strict that + base policy reads, because the run manufactures its own exception at install time; the failure is + measured, not hypothetical — a component installed while the environment was being created + carried egress past a global default-deny and origin data flowed. Until governance is configured + so component rules can only narrow, the surface is not `L2`-capable. - **`L3` — kernel-separated ephemeral environment.** Substrate classes (marked examples): a VM or microVM; a hosted ephemeral executor surface. Required where policy demands kernel separation — untrusted-provenance (`C5`) work. diff --git a/plugins/autonomy/skills/setup/SKILL.md b/plugins/autonomy/skills/setup/SKILL.md index f9cf1bab4e..2acb372a55 100644 --- a/plugins/autonomy/skills/setup/SKILL.md +++ b/plugins/autonomy/skills/setup/SKILL.md @@ -229,11 +229,17 @@ depends on the binding until that human-landed change exists. pre-existing surface is authoritative input to reconcile against, not a blank field to fill. 3. **Live-validate BEFORE recording** — the empirical probe per substrate class (recipe in [`templates/isolation-probe.md`](templates/isolation-probe.md)). A candidate `L2`/`L3` - substrate is validated by running, INSIDE the boundary, two probes that MUST both fail: - - a **denied-egress smoke test** — a network fetch to a well-known external host MUST fail - (a boundary that lets egress through is not an `L2` boundary); + substrate is validated by running, INSIDE the boundary, three probes that MUST all fail: + - a **denied-egress smoke test** — a network fetch MUST fail against two well-known external + hosts under different operators AND against every destination the level binding ratifies as + component-reachable (a boundary that lets egress through is not an `L2` boundary, and a probe + that samples only what the base policy denies never looks where an installed component may + already have widened it); - a **host-credential-path read attempt** — a read of a host credential path MUST be absent or - denied (a boundary that leaks host secrets is not an `L2` boundary). + denied (a boundary that leaks host secrets is not an `L2` boundary); + - a **workspace host-write containment check** — randomized canaries written inside MUST all be + absent on the host after teardown (a boundary the host later executes writes from is not an + `L2` boundary). The checker resolves no DNS and reads no remote host, so it validates the probe's targets against operator-configured seams. The egress target checks against `--egress-hosts ` (a @@ -256,8 +262,8 @@ depends on the binding until that human-landed change exists. lands ahead of the probe that proves its boundary. 4. **Bind level → substrate per surface** — record each validated substrate under its surface in `isolation_bindings` (surface id → level token → substrate instance + the human-ratified - `substrate_class` + `probe_evidence` + the non-forgeable `runtime_markers` the dispatch seam - attests against), plus the merge policy, + `substrate_class` and `component_reachable_hosts` + `probe_evidence` + the non-forgeable + `runtime_markers` the dispatch seam attests against), plus the merge policy, verification-blocking knobs, escalation routes, and admission rules and caps — all on the prepared security-binding change, validated by [`scripts/check-security-binding.mjs`](scripts/check-security-binding.mjs) against diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/admission-weakened-no-justification.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/admission-weakened-no-justification.json index 24d09ba05f..c0639f64ca 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/admission-weakened-no-justification.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/admission-weakened-no-justification.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ambiguous-runtime-markers.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ambiguous-runtime-markers.json index 0b3a3dcde7..5d8f77b1e0 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ambiguous-runtime-markers.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ambiguous-runtime-markers.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "pool": "blue", @@ -18,6 +21,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-b-l2.json", "runtime_markers": { "pool": "blue", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/as112-v6-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/as112-v6-valid.json index 046760ada6..0b9a6c62bf 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/as112-v6-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/as112-v6-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-as112-v6.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c2-auto-unratified.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c2-auto-unratified.json index 13f0bed8f5..78dc202d13 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c2-auto-unratified.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c2-auto-unratified.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-auto-unratified.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-auto-unratified.json index 7448441f25..fc1e96b6b0 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-auto-unratified.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-auto-unratified.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-blocking-unratified.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-blocking-unratified.json index 1400ebfcbe..e0827399c0 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-blocking-unratified.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c3-blocking-unratified.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c4-auto-merge.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c4-auto-merge.json index 792ec263ab..5b8744a170 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c4-auto-merge.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/c4-auto-merge.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/component-reach-empty-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/component-reach-empty-valid.json new file mode 100644 index 0000000000..6a53e60d6f --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/component-reach-empty-valid.json @@ -0,0 +1,95 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "component_reachable_hosts": [], + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/equal-specificity-tie.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/equal-specificity-tie.json index a67c3672e4..8cadb9f00a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/equal-specificity-tie.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/equal-specificity-tie.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-invalid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-invalid.json index beec7e83da..168b1bf960 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-invalid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-invalid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -61,11 +64,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-loop.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-loop.json index 76a7d9e18c..72440ba936 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-loop.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-ack-loop.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -61,11 +64,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-push-without-channel.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-push-without-channel.json index c19c5b547f..c3e53635da 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-push-without-channel.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-push-without-channel.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -50,7 +53,9 @@ "untrusted-provenance": "urgent" }, "escalation_severity_routes": { - "urgent": { "push": "personal-push:oncall-primary" } + "urgent": { + "push": "personal-push:oncall-primary" + } }, "admission": { "classification": { @@ -63,11 +68,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-runner-route-missing.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-runner-route-missing.json index aad5167663..d1abafbdfd 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-runner-route-missing.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-runner-route-missing.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -60,11 +63,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-bad-token.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-bad-token.json index 514a01c646..f0e2092aa3 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-bad-token.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-bad-token.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -60,11 +63,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-tracker-only-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-tracker-only-valid.json index 6a8bc1ef4f..f66de57b02 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-tracker-only-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-tracker-only-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -60,11 +63,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-valid.json index f05a19a1ec..511c55000e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/escalation-severity-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -56,8 +59,13 @@ "runner-cap-exceeded": "notice" }, "escalation_severity_routes": { - "attention": { "channel": "channel:autonomy-escalations" }, - "urgent": { "channel": "channel:autonomy-escalations", "push": "personal-push:oncall-primary" } + "attention": { + "channel": "channel:autonomy-escalations" + }, + "urgent": { + "channel": "channel:autonomy-escalations", + "push": "personal-push:oncall-primary" + } }, "escalation_ack": { "staleness_window": 72, @@ -74,11 +82,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/hosted-ephemeral-l3-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/hosted-ephemeral-l3-valid.json index 50899aa3e3..c3b7792c42 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/hosted-ephemeral-l3-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/hosted-ephemeral-l3-valid.json @@ -7,6 +7,9 @@ "L3": { "substrate": "hosted-ephemeral-pool", "substrate_class": "hosted-ephemeral-executor", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3-hosted.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/l3-ratified-class-container.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/l3-ratified-class-container.json index fb5c8b5264..5f35286700 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/l3-ratified-class-container.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/l3-ratified-class-container.json @@ -7,6 +7,9 @@ "L3": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3-container.json" } } @@ -53,11 +56,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/marker-prototype-key.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/marker-prototype-key.json index 7d609f9181..2946b350f7 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/marker-prototype-key.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/marker-prototype-key.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "toString": "l2" @@ -15,6 +18,9 @@ "L3": { "substrate": "microvm-pool", "substrate_class": "vm-microvm", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3.json", "runtime_markers": { "tier": "metal" @@ -64,11 +70,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-probe-evidence.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-probe-evidence.json index 77ac2aee57..9a5f90cbe9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-probe-evidence.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-probe-evidence.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "runtime_markers": { "runner-pool": "ci-pool-a", "region": "us-east" diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-substrate-class.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-substrate-class.json index 3e22b425b7..29c83dfcd4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-substrate-class.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/missing-substrate-class.json @@ -10,7 +10,10 @@ "runtime_markers": { "runner-pool": "ci-pool-a", "region": "us-east" - } + }, + "component_reachable_hosts": [ + "example.com" + ] } } }, @@ -56,11 +59,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/multi-host-egress-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/multi-host-egress-valid.json index d99310fd50..9edbe313e6 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/multi-host-egress-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/multi-host-egress-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-multi-host.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-relay.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-relay.json index 84aa0411a4..75f777e2c9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-relay.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-relay.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-6to4-relay.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-v6.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-v6.json index cd13d2e1fc..c7d1575cf4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-v6.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-6to4-v6.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-6to4-v6.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bare-metadata.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bare-metadata.json index aef19e9dec..9503c6676d 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bare-metadata.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bare-metadata.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-bare-metadata.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-benchmarking-v6.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-benchmarking-v6.json index e4a69ed5f1..3438d28bc4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-benchmarking-v6.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-benchmarking-v6.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-benchmarking-v6.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bogus-metadata.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bogus-metadata.json index e2a5aeba0b..1576dd26aa 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bogus-metadata.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-bogus-metadata.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-bogus-metadata.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-partial.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-partial.json new file mode 100644 index 0000000000..2c374f5f54 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-partial.json @@ -0,0 +1,98 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "component_reachable_hosts": [ + "example.com", + "api.example.net" + ], + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-uncovered.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-uncovered.json new file mode 100644 index 0000000000..16bbdf4c99 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-uncovered.json @@ -0,0 +1,98 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "component_reachable_hosts": [ + "api.example.net", + "cdn.example.net" + ], + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-unratified.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-unratified.json new file mode 100644 index 0000000000..738190de91 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-component-reach-unratified.json @@ -0,0 +1,94 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "human", + "C3": "human", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "not-required", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-missing.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-missing.json index cce7464d80..3ddc806cc7 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-missing.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-missing.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-cred-outer-missing.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-nonzero.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-nonzero.json index 39f9ba8e6c..23b89caee9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-nonzero.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-cred-outer-nonzero.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-cred-outer-nonzero.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-code-count.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-code-count.json index e1e626f999..4fd982d184 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-code-count.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-code-count.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-credpath-code-count.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-partial-zero.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-partial-zero.json index 3184fb7e01..ec7073d06f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-partial-zero.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credpath-partial-zero.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-credpath-partial-zero.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-case-mismatch.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-case-mismatch.json index 83fdc1f619..e83a105a1b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-case-mismatch.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-case-mismatch.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-id-rsa.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-drive-root.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-drive-root.json index 0113ac1b66..5668c50d7c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-drive-root.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-drive-root.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-drive-root.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-relative-root.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-relative-root.json index 74aa9be377..1b05cfe364 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-relative-root.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-relative-root.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-relative-credpath.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-root-slash.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-root-slash.json index 83fdc1f619..e83a105a1b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-root-slash.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-root-slash.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-id-rsa.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-unconfigured.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-unconfigured.json index 83fdc1f619..e83a105a1b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-unconfigured.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-credroots-unconfigured.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-id-rsa.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-discard-only.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-discard-only.json index 600bf37011..9860d80127 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-discard-only.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-discard-only.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-discard-only.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-doc-3fff.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-doc-3fff.json index 7b117bc81c..a95d7ccc21 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-doc-3fff.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-doc-3fff.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-doc-3fff.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-docnet-host.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-docnet-host.json index d3fc7513ba..4d8225da6b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-docnet-host.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-docnet-host.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-docnet.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dot-traversal.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dot-traversal.json index 4b3f8c025e..6e319da024 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dot-traversal.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dot-traversal.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-dot-traversal.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dummy-prefix.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dummy-prefix.json index 2471904cd7..6d5034269a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dummy-prefix.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-dummy-prefix.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-dummy-prefix.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-client-not-ready.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-client-not-ready.json index 154c1bf9ee..273651d431 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-client-not-ready.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-client-not-ready.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-client-not-ready.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-duplicate-host.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-duplicate-host.json index 81401121b9..44f4740576 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-duplicate-host.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-duplicate-host.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-duplicate-host.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-fingerprint-on-nonpeer.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-fingerprint-on-nonpeer.json index 9d8b4eed24..2d0b83f8a3 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-fingerprint-on-nonpeer.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-fingerprint-on-nonpeer.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-fingerprint-on-nonpeer.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-no-transport.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-no-transport.json index 7dddbe306d..9a366ee2bd 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-no-transport.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-no-transport.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-no-transport.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-peer-match.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-peer-match.json index 8cd2a48b6d..254656eba2 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-peer-match.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-peer-match.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-peer-match.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-single-target.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-single-target.json index 510e41372f..fe31d6aa1b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-single-target.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-single-target.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-single-target.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-count.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-count.json index 171d7b7ee8..a9aa655e6c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-count.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-count.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-transport-count.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-token.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-token.json index 5c1545d126..4cd6b49c8e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-token.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-transport-token.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-transport-token.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-zero-exit-peer-substituted.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-zero-exit-peer-substituted.json index 0a6c457f4f..adac1a5924 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-zero-exit-peer-substituted.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-egress-zero-exit-peer-substituted.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-egress-zero-exit-peer-substituted.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-encoded-loopback.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-encoded-loopback.json index 117f27e742..1f80908119 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-encoded-loopback.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-encoded-loopback.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-encoded-loopback.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expanded-loopback.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expanded-loopback.json index 7beba7d36c..f40c91a0bf 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expanded-loopback.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expanded-loopback.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-expanded-loopback.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-ephemeral.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-ephemeral.json index c0e594a251..7dddfd2ae1 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-ephemeral.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-ephemeral.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-expansion-ephemeral.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-tail-mismatch.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-tail-mismatch.json index d8c72332e2..1115c9ca6a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-tail-mismatch.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-expansion-tail-mismatch.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-expansion-tail-mismatch.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-generic-env-token.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-generic-env-token.json index 7d091c2e91..7d896121c4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-generic-env-token.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-generic-env-token.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-generic-env-token.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-host-home-mount.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-host-home-mount.json index 60d56759c9..099e7b8173 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-host-home-mount.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-host-home-mount.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-host-home-mount.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-imds-http-error.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-imds-http-error.json index 3a86bb3082..401597327e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-imds-http-error.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-imds-http-error.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-imds-http-error.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invalid-tld.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invalid-tld.json index bbdb41bf63..d90af7872b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invalid-tld.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invalid-tld.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-invalid-tld.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-env-token.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-env-token.json index 5c85374ac1..4a71aafb3f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-env-token.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-env-token.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-invented-env-token.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-home-user.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-home-user.json index 99dde248b2..343982d0f0 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-home-user.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-invented-home-user.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-invented-home-user.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ipv4-compat.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ipv4-compat.json index b90d17d326..ad66cfb74e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ipv4-compat.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ipv4-compat.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-ipv4-compat.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-l3-container.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-l3-container.json index 6714196c03..ea4146608e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-l3-container.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-l3-container.json @@ -7,6 +7,9 @@ "L3": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3-container.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-level-mismatch.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-level-mismatch.json index 2273e68d5a..114e399b7b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-level-mismatch.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-level-mismatch.json @@ -7,6 +7,9 @@ "L3": { "substrate": "egress-denied-container-pool", "substrate_class": "vm-microvm", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-local-host.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-local-host.json index b6b858e653..12db323e6e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-local-host.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-local-host.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-localhost.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-malformed-dns.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-malformed-dns.json index f9abba05b6..768d57f106 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-malformed-dns.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-malformed-dns.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-malformed-dns.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-expansion-mismatch.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-expansion-mismatch.json index dfc7b2c534..bb2fd0bd0b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-expansion-mismatch.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-expansion-mismatch.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-metadata-expansion-mismatch.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-port.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-port.json index af7706faf9..e84819ac19 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-port.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-port.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-metadata-port.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route-boundary.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route-boundary.json index 4a16eb540a..59f7ea9da1 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route-boundary.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route-boundary.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-metadata-route-boundary.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route.json index 279eb5d2db..829a3d0c3f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-route.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-metadata-route.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-scheme.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-scheme.json index 8f7a85981c..e5ec53d44d 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-scheme.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-metadata-scheme.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-metadata-scheme.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-host-expanded.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-host-expanded.json index fcdb7158a6..2bd11551e2 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-host-expanded.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-host-expanded.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-no-host-expanded.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-outer-exit.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-outer-exit.json index 038fa8dc3d..3b88353c24 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-outer-exit.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-outer-exit.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-no-outer-exit.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-probed-at.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-probed-at.json index ed9b219cbf..0d1764b5b3 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-probed-at.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-probed-at.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-no-probed-at.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-target.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-target.json index 066dbd741f..f34ad32e6a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-target.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-target.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-no-host.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-transport.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-transport.json index edaae30e93..ece1579f40 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-transport.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-missing-transport.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-no-transport.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-multi-host-private.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-multi-host-private.json index f571389bc7..7873c13d2a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-multi-host-private.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-multi-host-private.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-multi-host-private.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-local.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-local.json index 6e371ee9cb..2c70d7e845 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-local.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-local.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-nat64-local.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-private-embed.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-private-embed.json index cd06d8d946..17f57ed37b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-private-embed.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-nat64-private-embed.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-nat64-private-embed.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-outer-egress-failed.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-outer-egress-failed.json index 34e64b5ac0..8bc9c88993 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-outer-egress-failed.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-outer-egress-failed.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-outer-egress-failed.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-protocol-assignment.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-protocol-assignment.json index a8697cc9ef..ecb2cd356b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-protocol-assignment.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-protocol-assignment.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-protocol-assignment.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ratified-class-mismatch.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ratified-class-mismatch.json index 43b3e5e940..670c48c11c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ratified-class-mismatch.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-ratified-class-mismatch.json @@ -7,6 +7,9 @@ "L3": { "substrate": "egress-denied-container-pool", "substrate_class": "vm-microvm", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3-container.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-root-escape.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-root-escape.json index 4d6c7864b2..e8a1b7148b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-root-escape.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-root-escape.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "../ci-pool-a-l2-outside-root.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-site-local-v6.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-site-local-v6.json index 9316c4fe9a..80ab080d23 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-site-local-v6.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-site-local-v6.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-site-local-v6.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-srv6-sid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-srv6-sid.json index d5f834b76a..18555b28a3 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-srv6-sid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-srv6-sid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-srv6-sid.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-teredo.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-teredo.json index 927b00eab7..8b96ebf808 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-teredo.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-teredo.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-teredo.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-transport-cross-type.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-transport-cross-type.json index 8d2bea909a..7b42637c8e 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-transport-cross-type.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-transport-cross-type.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-transport-cross-type.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-unc-credpath.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-unc-credpath.json index 7015955eda..aff41c0771 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-unc-credpath.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-unc-credpath.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-unc-credpath.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-uri-host.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-uri-host.json index c1d8c7f34d..fdb6ad77f9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-uri-host.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-uri-host.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-uri-host.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-count.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-count.json index 7470124b5e..299100373c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-count.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-count.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-canary-count.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-duplicate.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-duplicate.json index 2fed013d75..0e4a6eb7f8 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-duplicate.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-duplicate.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-canary-duplicate.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-shapes.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-shapes.json index 5a44791af2..a5819d5820 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-shapes.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-shapes.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-canary-shapes.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-traversal.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-traversal.json index d0fe7477ef..692d05504b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-traversal.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-traversal.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-canary-traversal.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-visible.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-visible.json index 5035c2f738..0138f88270 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-visible.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-canary-visible.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-canary-visible.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-git-digest-changed.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-git-digest-changed.json index dc1a5f0c2c..6ecce63499 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-git-digest-changed.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-git-digest-changed.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-git-digest-changed.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-inner-code-empty.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-inner-code-empty.json index 29b5c91a37..831537a7c9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-inner-code-empty.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-inner-code-empty.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-inner-code-empty.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-missing.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-missing.json index 1e1be27ac7..4c508d6fa3 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-missing.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-missing.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-missing.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-no-teardown.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-no-teardown.json index e284f65146..ab0e13d729 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-no-teardown.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-no-teardown.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-no-teardown.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-not-applicable.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-not-applicable.json index 25438a0ed9..bde68a23d8 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-not-applicable.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-workspace-not-applicable.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-workspace-not-applicable.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-zero-exit.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-zero-exit.json index 2ad14205cf..08d2ca1ab7 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-zero-exit.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/probe-evidence-zero-exit.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-zero-exit.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2.json index 13c6d69fd3..dd08c69649 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json index b64d6cf6d7..828e93f629 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json index cccdc37a9e..2cbaef1627 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json index 043a48e867..b6bdd5970a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json index cc94390b2c..c7187566c1 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promotion-nonpromotable.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promotion-nonpromotable.json index 00150f75bd..d3f697262f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promotion-nonpromotable.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promotion-nonpromotable.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-calendar-invalid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-calendar-invalid.json index 5ffe580bca..3fd98ce122 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-calendar-invalid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-calendar-invalid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -65,11 +68,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-non-iso.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-non-iso.json index 8d4ea95721..37bc14864a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-non-iso.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-non-iso.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-offset-invalid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-offset-invalid.json index f924f1d1e6..80c8c044de 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-offset-invalid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-offset-invalid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -65,11 +68,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-submilli.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-submilli.json index 6e433cf435..9cc4ef0989 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-submilli.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ratified-at-submilli.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -65,11 +68,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/same-surface-level-ambiguity.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/same-surface-level-ambiguity.json index d90cb36efc..22d95da71a 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/same-surface-level-ambiguity.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/same-surface-level-ambiguity.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -16,6 +19,9 @@ "L3": { "substrate": "microvm-pool", "substrate_class": "vm-microvm", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ssh-id-rsa-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ssh-id-rsa-valid.json index 83fdc1f619..e83a105a1b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ssh-id-rsa-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/ssh-id-rsa-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2-id-rsa.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-bare-string.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-bare-string.json index 1a6a9d121f..51d961cfd4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-bare-string.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-bare-string.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -66,11 +69,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-valid.json index 88d8ad22d7..3b6053155f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-classification-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-duplicate-producer-identity.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-duplicate-producer-identity.json index 8b562dc579..64848080ac 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-duplicate-producer-identity.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-duplicate-producer-identity.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-empty-source-surface.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-empty-source-surface.json index 8c71a49e30..540752b3c6 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-empty-source-surface.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-empty-source-surface.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-identity-grammar.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-identity-grammar.json index b501259e02..d240e3d4cb 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-identity-grammar.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-identity-grammar.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-producer-identity.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-producer-identity.json index e9c6e4be34..b3867ff42b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-producer-identity.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-producer-identity.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -70,11 +73,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-run-link-prefix.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-run-link-prefix.json index 3ba58bf7ea..e66a4c218c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-run-link-prefix.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-missing-run-link-prefix.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -70,11 +73,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-run-link-prefix-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-run-link-prefix-valid.json index 5119482ac4..6654d683a6 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-run-link-prefix-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-run-link-prefix-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-source-surface.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-source-surface.json index a6265ad25c..fa62316f45 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-source-surface.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/temporal-shared-source-surface.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -71,11 +74,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json index 5c5c7da77c..722e392ad5 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-all-deterministic-c4.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json index d44e5d8244..8e7eb7f426 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-checker-count-below-min.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json index 3420c0755e..6be6f1f60d 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-decorative.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json index c91db0021c..1327a98faf 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-dropped.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json index e2062fc2fe..46923883e2 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-no-generator.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json index 8fb3beb1b3..059c6be155 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-cross-vendor-vacuous.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json index be09739665..683ae6c8bd 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-modality.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json index 7fc2b3472a..5878ad89e9 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-relation-target.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json index ecfc20802e..83e92576f5 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-duplicate-role-name.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json index 3d0a0726d0..e2b8b27a77 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-min-checkers-weakened.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json index d9267bc7ae..738758a8b4 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-non-class-key.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json index 8d96f151c1..ae36740b5d 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-self-reference.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json index a14071c539..e78c032deb 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-tightened-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json index 0827bf013b..4d3512302d 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unconstrained-checker-pair.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json index 4f9fb9bdd8..1fd3716687 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-undeclared-role.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json index 0d5d6c2339..48c9694b46 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/topology-unknown-predicate.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/two-level-surface-valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/two-level-surface-valid.json index cd26ad8223..e8f5ca694f 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/two-level-surface-valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/two-level-surface-valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -16,6 +19,9 @@ "L3": { "substrate": "microvm-pool", "substrate_class": "vm-microvm", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l3.json", "runtime_markers": { "runner-pool": "ci-pool-a-metal", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/unverifiable-probe-evidence.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/unverifiable-probe-evidence.json index 66278d5a79..c5d691222b 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/unverifiable-probe-evidence.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/unverifiable-probe-evidence.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/does-not-exist.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/valid.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/valid.json index 324b9fe4a2..ee1e925217 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/valid.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/valid.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", @@ -57,11 +60,36 @@ } }, "rules": [ - { "signal_class": "*", "provenance": "*", "work_class": "C1", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C2", "disposition": "autonomous-eligible" }, - { "signal_class": "*", "provenance": "*", "work_class": "C3", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C4", "disposition": "human-gated" }, - { "signal_class": "*", "provenance": "*", "work_class": "C5", "disposition": "human-gated" } + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } ], "autonomous_concurrency": 1, "items_per_run": 3 diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/vendor-hosted-auto.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/vendor-hosted-auto.json index 05ecdc08e3..a9de8699fd 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/vendor-hosted-auto.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/vendor-hosted-auto.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/verification-below-floor.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/verification-below-floor.json index 6d1d1d410e..d65151c501 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/verification-below-floor.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/verification-below-floor.json @@ -7,6 +7,9 @@ "L2": { "substrate": "egress-denied-container-pool", "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", "runtime_markers": { "runner-pool": "ci-pool-a", diff --git a/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json b/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json index d67efd8eac..4d1f1bf9c0 100644 --- a/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json +++ b/plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json @@ -186,8 +186,13 @@ "description": "The HUMAN-RATIFIED substrate-class assertion the eligibility decision keys off, living here on the agent-unwritable surface — never resolved from the probe transcript, whose recorded class is capture evidence the executing agent could doctor and must EQUAL this value to verify. An L3 entry ratifying a class outside the kernel-separated pair (vm-microvm, hosted-ephemeral-executor) is an invalid binding, fail-closed.", "enum": ["container", "os-sandbox", "vm-microvm", "hosted-ephemeral-executor"] }, + "component_reachable_hosts": { + "description": "The HUMAN-RATIFIED set of external destinations this surface's installed components may request — an outer-world fact no capture can establish, so it lives here on the agent-unwritable surface rather than in the transcript. An L2/L3 probe transcript verifies only when its probed egress targets cover this set in FULL: each ratified destination is a separate policy decision, so covering one says nothing about the rest. The EMPTY list is the explicit claim that this surface installs nothing carrying policy rules of its own; an ABSENT field leaves the level UNPROVEN and excluded from isolation eligibility, fail-closed.", + "type": "array", + "items": { "type": "string", "minLength": 1 } + }, "probe_evidence": { - "description": "Reference to the live-validation probe transcript that proved this boundary; a binding without probe evidence is invalid. For L2/L3 the referenced transcript must VERIFY (capture shape with all three assertions — denied egress proven by peer identity, absent host credentials, contained workspace host-writes — and a networked outer context) — under autonomous-enabled an unverifiable transcript leaves the level UNPROVEN and excluded from isolation eligibility, fail-closed. A transcript captured before the workspace assertion existed does not verify: the level goes UNPROVEN and the surface re-probes.", + "description": "Reference to the live-validation probe transcript that proved this boundary; a binding without probe evidence is invalid. For L2/L3 the referenced transcript must VERIFY (capture shape with all three assertions — denied egress proven by peer identity, absent host credentials, contained workspace host-writes — and a networked outer context) — under autonomous-enabled an unverifiable transcript leaves the level UNPROVEN and excluded from isolation eligibility, fail-closed. A transcript captured before the workspace assertion existed does not verify: the level goes UNPROVEN and the surface re-probes. The transcript must additionally cover every ratified component_reachable_hosts destination.", "type": "string", "minLength": 1 }, diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json index 50cb7cb3b5..8397af0594 100644 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json @@ -942,6 +942,32 @@ "verification_topology.C3.roles[1]: unknown key \"max_latency_ms\" (additionalProperties: false)" ], "credential_roots": "/home/runner" + }, + "probe-evidence-component-reach-uncovered.json": { + "exit": 1, + "findings_substrings": [ + "leaving ratified component_reachable_hosts api.example.net, cdn.example.net unprobed" + ], + "credential_roots": "/home/runner" + }, + "probe-evidence-component-reach-partial.json": { + "exit": 1, + "findings_substrings": [ + "leaving ratified component_reachable_hosts api.example.net unprobed" + ], + "credential_roots": "/home/runner" + }, + "probe-evidence-component-reach-unratified.json": { + "exit": 1, + "findings_substrings": [ + "the level binding ratifies no component_reachable_hosts" + ], + "credential_roots": "/home/runner" + }, + "component-reach-empty-valid.json": { + "exit": 0, + "findings_substrings": [], + "credential_roots": "/home/runner" } }, "quarantined": {}, diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs index 4700db806c..6685d964cb 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs @@ -546,7 +546,14 @@ function validateStructure(binding) { findings.push(`${where}: must be an object`); continue; } - checkAllowedKeys(entry, ["substrate", "substrate_class", "probe_evidence", "runtime_markers"], where); + checkAllowedKeys(entry, ["substrate", "substrate_class", "component_reachable_hosts", "probe_evidence", "runtime_markers"], where); + if ( + Object.hasOwn(entry, "component_reachable_hosts") && + (!Array.isArray(entry.component_reachable_hosts) || + entry.component_reachable_hosts.some((host) => !isNonEmptyString(host))) + ) { + findings.push(`${where}.component_reachable_hosts: must be an array of non-empty host strings — the empty array is the explicit claim that this surface installs nothing carrying policy rules of its own`); + } if (!isNonEmptyString(entry.substrate)) { findings.push(`${where}.substrate: missing or empty — the bound substrate instance id is required`); } @@ -1241,7 +1248,7 @@ function isNonExternalEgressHost(host) { // surface, level, substrate, or substrate class proves a DIFFERENT boundary, // not this one. Returns null when verified, else the reason the entry is // unproven. -function verifyProbeTranscript(ref, probeRoot, surfaceId, level, substrate, substrateClass, egressAllowList, credentialRoots) { +function verifyProbeTranscript(ref, probeRoot, surfaceId, level, substrate, substrateClass, egressAllowList, credentialRoots, componentReachableHosts) { // Evidence verifies ONLY against the configured protected root: without // --probe-evidence-root a ref resolves as written — including to an // agent-writable file swapped after the human ratified the binding — so no @@ -1537,7 +1544,7 @@ function verifyProbeTranscript(ref, probeRoot, surfaceId, level, substrate, subs return `transcript ${path} records assertions.egress_denied.client_ready ${JSON.stringify(egress.client_ready)} — the probe client must first be shown to RUN inside the boundary (exit "0" against an in-boundary endpoint): an absent or broken client would otherwise satisfy every egress assertion trivially`; } // One denied destination is fully consistent with a policy that allows - // others — including one a kit installed on top of a global deny-all. + // others — including one a component installed on top of a global deny-all. if (egressHosts.length < 2) { return `transcript ${path} records ${egressHosts.length} probed egress target — a single probed target cannot establish default-deny egress, since a policy may allow other destinations while denying this one; probe at least two distinct external targets under different operators`; } @@ -1593,6 +1600,32 @@ function verifyProbeTranscript(ref, probeRoot, surfaceId, level, substrate, subs } const workspaceProblem = verifyWorkspaceContainment(transcript, path); if (workspaceProblem !== null) return workspaceProblem; + // Target selection is load-bearing, and this leg is LAST for the reason the + // whole hardened block is: the function returns the FIRST problem, and every + // pre-existing fixture pins the reason its own transcript is rejected for. + // + // The distinct-targets leg above samples only what the BASE policy denies. A + // surface whose additive policy layer lets an installed component carry rules + // of its own is widened at exactly the destinations those components request, + // so a probe drawn from anywhere else certifies a boundary open at the one + // place it never looked. Which destinations those are is an outer-world fact + // no capture can establish, so the set rides the HUMAN-RATIFIED level entry on + // the agent-unwritable surface — the trust boundary substrate_class sits on — + // and the transcript's job is to show the probe covered it. + if (!Array.isArray(componentReachableHosts)) { + return `the level binding ratifies no component_reachable_hosts — the destinations this surface's installed components may request are an outer-world fact no capture can establish, so they are ratified on the binding entry and the probe must cover them; ratify the list (or the empty list, where this surface installs nothing carrying policy rules of its own) on the agent-unwritable binding surface`; + } + if (componentReachableHosts.some((host) => !isNonEmptyString(host))) { + return `the level binding ratifies a component_reachable_hosts entry that is not a non-empty host string — every ratified destination must name a host the probe can be checked against`; + } + // Coverage, never a count: each ratified destination is a separate policy + // decision, so probing one says nothing about the rest. + const uncoveredRatified = componentReachableHosts + .map((host) => host.trim().toLowerCase().replace(/\.$/, "")) + .filter((host) => !distinctEgressHosts.has(host)); + if (uncoveredRatified.length > 0) { + return `transcript ${path} probes ${[...distinctEgressHosts].join(", ")}, leaving ratified component_reachable_hosts ${uncoveredRatified.join(", ")} unprobed — each ratified destination is a separate policy decision, so covering one says nothing about the rest; probe every ratified component-reachable destination in the configuration the run will actually use`; + } return null; } @@ -1787,7 +1820,7 @@ function checkSemantics(binding, probeRoot, egressAllowList, credentialRoots) { continue; } const reason = isNonEmptyString(entry.probe_evidence) - ? verifyProbeTranscript(entry.probe_evidence, probeRoot, surfaceId, level, entry.substrate, entry.substrate_class, egressAllowList, credentialRoots) + ? verifyProbeTranscript(entry.probe_evidence, probeRoot, surfaceId, level, entry.substrate, entry.substrate_class, egressAllowList, credentialRoots, entry.component_reachable_hosts) : "probe_evidence missing"; if (reason === null) { provenMax = Math.max(provenMax, levelNo); diff --git a/plugins/autonomy/skills/setup/templates/isolation-probe.md b/plugins/autonomy/skills/setup/templates/isolation-probe.md index 1d8f129a45..c1fd849f7b 100644 --- a/plugins/autonomy/skills/setup/templates/isolation-probe.md +++ b/plugins/autonomy/skills/setup/templates/isolation-probe.md @@ -15,7 +15,7 @@ Three checks, run inside the boundary, all expected to FAIL: | Assertion | Runs | Expected result | |---|---|---| -| Denied egress | a TLS fetch of two `` targets under different operators | no origin peer answered — NON-zero exit, and no in-boundary peer identity matching the outer context's | +| Denied egress | a TLS fetch of two `` targets under different operators, plus every destination the level binding ratifies as component-reachable | no origin peer answered — NON-zero exit, and no in-boundary peer identity matching the outer context's | | Absent host credentials | a read of `` | file absent, or read denied — NON-zero exit | | Contained workspace host-writes | randomized canary writes into the workspace mount, re-checked on the host after teardown | every canary still absent on the host, and the VCS control-plane digest unchanged | @@ -51,7 +51,7 @@ trusted-root values bind per the deployment's secret-binding classification. ## Egress-denial probe shape -Three properties have to hold together, because each one alone is satisfiable by a boundary that is +Four properties have to hold together, because each one alone is satisfiable by a boundary that is not `L2`. **The client must be shown to run.** A boundary with no working fetch tool denies nothing, yet a @@ -59,13 +59,37 @@ missing tool is indistinguishable from a sealed network by exit code alone — s boundary would otherwise score best. Prove the client runs before believing anything it reports. **At least two targets, under different operators.** One denied destination is fully consistent with -a policy that allows others; a kit installed on top of a global deny-all can add its own allow rule. +a policy that allows others; a component installed on top of a global deny-all can add its own allow +rule. **Each target must be reachable from the OUTER context first.** A target that fails everywhere — an unregistered name, a dead host — "fails" inside too and proves nothing. This is why the targets are well-known hosts rather than unguessable ones: unguessability and outer-reachability cannot both hold over DNS, and outer-reachability is what makes the inner failure mean something. +**Every destination the level binding ratifies as component-reachable must be among the targets.** +Two targets under different operators sample only what the BASE policy denies. Where the surface +carries an additive policy layer, the destinations its installed components request are the exact +places the boundary may already have been widened, so a probe drawn from anywhere else certifies a +boundary open at the one place it never looked. Which destinations those are is an outer-world fact +no capture can establish, so the set is HUMAN-RATIFIED on the level binding's +`component_reachable_hosts` — the agent-unwritable surface `substrate_class` already sits on — and +the probe covers it in FULL, since covering one ratified destination says nothing about the rest. +An empty ratified list is the explicit claim that this surface installs nothing carrying policy +rules of its own; an ABSENT one leaves the level unproven, fail-closed. Ratify only destinations +the outer context can itself reach: a component-reachable destination on a private or internal +network fails the outer-reachability property above and is outside what this probe settles. Probe +in the configuration the run will ACTUALLY use, those components installed. + +What this settles and what it does not. A transcript can prove that the probe covered every +destination the human ratified, and the egress assertion then proves each one was denied. It cannot +prove the ratified set is COMPLETE — a component requesting a destination nobody ratified is +invisible to every capture — and it cannot prove the probe ran with those components installed at +all. The first is where the human takes responsibility by ratifying on the agent-unwritable surface; +nothing takes responsibility for the second, and a component-reachable destination on a private or +internal network is outside the probe's reach entirely, since a non-external target cannot evidence +external egress denial. These are RECORDED, not implied. + ```sh # readiness (inside the boundary): the client itself must work — record as client_ready ; test $? -eq 0 || fail "the probe client does not run inside the boundary — an absent client would satisfy every egress assertion trivially" From e57797d5eef6affb94879fbf0f695ef83ee44769 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 23:13:59 -0400 Subject: [PATCH 15/17] feat(autonomy): reject a binding that can auto-merge with no force behind its checkers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The brief located this check on the verification-topology floor, and that check cannot be built. The floors are tighten-only and floor-bounded, so no value a binding can set describes a topology incapable of unanimity — the check would never fire and its fixture pair could never be constructed. Dead code with a green suite is worse than no check, so the enforcement sits where the failure actually lives. Unanimity needs two things: a checker POPULATION and FORCE behind its verdicts. The topology floors supply the population; the security-review blocking knob supplies the force. Only force is configurable into absence, so the rule is a JOIN — each axis alone at a perfectly legal value still combines into an automatic transition no checker can withhold. An advisory layer records a dissent and proceeds anyway. A not-required layer with model-adjudicated checkers declared means the layer those checkers judge in never runs, so their agreement can never be obtained and unanimity over them is vacuous. PROMOTION_DEPENDENCIES gains the review cell because the forbidden state was reachable dynamically, through the exact resolution the checker header tells consumers to perform before every merge decision: contrary evidence demotes C3 ai-review-blocking back to its advisory floor while C3 auto-merge stays effective-promoted. Static validity alone did not cover it. The cell is appended rather than prepended — `failed` is built in dependency-array order and a fixture pins the C2 cell by name. Four existing bindings encoded auto-merge over an advisory review layer and are migrated rather than grandfathered. That is the intended bar raise, the same shape the isolation phase accepted for adopters, and it belongs in the release note rather than absorbed silently. The contract records what ships unverified rather than implying otherwise. Per-run aggregation — unanimous pass, single dissent, checker timeout, duplicate checker identity — is a runner-seam obligation, and no runner exists to exercise it. Resolved distinctness is the same: the rule is stated over the binding because a binding is what a check can read, and two slots held distinct by declared constraints can still resolve to one instance at run time. Dissent routes on the existing verification-divergence class. No token, no second channel. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- plugins/autonomy/reference/guardrails.md | 9 ++ .../guardrails/verification-topology.md | 50 ++++++++ .../reference/guardrails/work-classes.md | 10 +- .../auto-merge-advisory-checkers.json | 117 ++++++++++++++++++ .../auto-merge-blocking-checkers.json | 117 ++++++++++++++++++ .../security-binding/promoted-c2c3-late.json | 8 +- .../security-binding/promoted-c2c3.json | 8 +- .../promoted-c2reearned-c3.json | 8 +- .../security-binding/promoted-c3-only.json | 8 +- ...curity-binding.fixtures.test.manifest.json | 13 ++ .../setup/scripts/check-security-binding.mjs | 60 ++++++++- 11 files changed, 402 insertions(+), 6 deletions(-) create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-advisory-checkers.json create mode 100644 plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-blocking-checkers.json diff --git a/plugins/autonomy/reference/guardrails.md b/plugins/autonomy/reference/guardrails.md index a8854c43c0..4fb19ce155 100644 --- a/plugins/autonomy/reference/guardrails.md +++ b/plugins/autonomy/reference/guardrails.md @@ -70,6 +70,15 @@ per-action prompts remain the control; at `L2` and above the whole-process bound control, replacing per-action prompts. The [isolation-ladder](guardrails/isolation-ladder.md) leaf carries this note in context. +## Automatic transitions + +Unanimity is not a matrix column: every transition a run takes without a human — merge and +every stage before it — requires unanimous agreement among the checkers the class declares, +and one dissent hands the item to the human gate and raises `verification-divergence` below. +It is a fixed invariant, not a knob, so no cell above can express it. The +[verification-topology](guardrails/verification-topology.md) leaf carries it in context, +including what is enforced at binding-validity time today and what is deferred. + ## Escalation Six escalation event classes: diff --git a/plugins/autonomy/reference/guardrails/verification-topology.md b/plugins/autonomy/reference/guardrails/verification-topology.md index 8f9b30087c..65a7614e8c 100644 --- a/plugins/autonomy/reference/guardrails/verification-topology.md +++ b/plugins/autonomy/reference/guardrails/verification-topology.md @@ -146,3 +146,53 @@ reading as if it meant more. human — not merge alone — requires every checker the class declares to agree. One dissent withholds the automatic transition and hands the item to the human gate; divergence routing is owned by the matrix's escalation contract. + +## How unanimity is enforced today + +Unanimity needs a checker POPULATION and FORCE behind its verdicts. This leaf's floors supply the +population; the [security-review leaf](security-review.md)'s per-class blocking knob supplies the +force. Only force is configurable into absence — floors are tighten-only, so no floor value can +describe a topology that cannot be unanimous, while a knob left below `blocking` lets a dissent be +recorded and the transition proceed anyway. + +**Binding-validity rule.** A class whose merge disposition is bound `auto` is INVALID, rejected at +check time, when either holds: + +- any verification layer for that class is bound `advisory` — the checker runs, dissents, and the + transition proceeds regardless; or +- the class declares a model-adjudicated checker slot while its model-adjudicated layer is bound + `not-required` — the layer that slot judges in never runs, so its agreement can never be obtained + and unanimity over it is vacuous. + +A class declaring no model-adjudicated slot is not caught by the second case: its floor is seated by +a deterministic slot, whose force is its own layer. The rule is a JOIN across two axes, never a floor +on either — each axis alone at a legal value can still combine into an automatic transition no +checker can withhold. + +**Dissent routes on the existing channel.** A withheld transition files on the bound route for the +matrix's `verification-divergence` event class, whose definition already covers checker +disagreement. No token and no channel is added here; the one-channel invariant binds unchanged. + +**Consumers resolve both.** A consumer resolving a class's merge disposition must resolve its +checker layers in the same step, against live promotion-evidence telemetry: automatic merge is +unavailable whenever a checker layer resolves below `blocking`, including where a human-ratified +`blocking` has been lowered by automatic demotion. Reading the bound merge disposition alone is +non-conforming. + +**Why the check is merge-scoped while the obligation is not.** The obligation covers every +transition a run takes without a human. Merge is the only such transition a binding can express: +intermediate pipeline transitions are runner-owned, and no runner exists. Autonomous ADMISSION is +not a second hole — admission precedes the artifact, so there is no checker verdict to be unanimous +about at that point. + +**Two limits, stated rather than hidden.** Neither is verified anywhere today. + +- **Per-run aggregation is not asserted.** Unanimous pass, single dissent, checker timeout, and + duplicate checker identity at run time are verdict-aggregation obligations on the runner seam, + deferred to the runner's build trigger. What ships is the contract obligation and the + binding-validity rule above: a configuration that could auto-proceed with no force behind its + checkers is rejected; a RUN that does so is not yet detectable. +- **Force is checked; RESOLVED distinctness is not.** The slot rule above is stated over the binding + because a binding is what a check can read. Two slots held distinct by declared constraints can + still resolve to one instance at run time, and no static check sees that — it is the same + runner-seam obligation. diff --git a/plugins/autonomy/reference/guardrails/work-classes.md b/plugins/autonomy/reference/guardrails/work-classes.md index e8463b338e..8cc7aa0eca 100644 --- a/plugins/autonomy/reference/guardrails/work-classes.md +++ b/plugins/autonomy/reference/guardrails/work-classes.md @@ -60,6 +60,12 @@ per the [telemetry contract](../telemetry.md) are the evidence base. - **Demotion is automatic and fail-closed.** Contrary evidence lowers the cell's effective state immediately, without waiting for human action; the cell re-earns promotion from there through the same evidence predicate. +- **Promotion never overrides unanimity.** A promoted `C2`/`C3` auto-merge cell still does not + auto-proceed on checker dissent: the promoted state is a ceiling, and dissent withholds the + automatic transition the same way contrary evidence lowers the cell — the same mechanism, not + a second one beside it. Requiring unanimous checker agreement is a + [verification-topology](verification-topology.md) invariant, never a promotable knob; that + leaf states what is checked at binding-validity time and what awaits the runner. ## Suggested default predicates @@ -76,4 +82,6 @@ org binds (org-bindable values): **Demotion evidence set** (one event suffices): any post-merge gate failure, any human-reverted merge, any verification divergence. Demotion cascades along predicate dependencies: `C3` auto-merge is earned on the `C2` auto-merge track -record, so contrary evidence against `C2` auto-merge demotes both cells. +record, and its automatic transition is gated on the `C3` AI-review cell being +blocking, so contrary evidence against either prerequisite demotes `C3` +auto-merge with it. diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-advisory-checkers.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-advisory-checkers.json new file mode 100644 index 0000000000..45596feab4 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-advisory-checkers.json @@ -0,0 +1,117 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "auto", + "C3": "auto", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "advisory", + "C3": "advisory", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + }, + "promotion_state": { + "C2-auto-merge": { + "state": "promoted", + "ratified_by_change": "change/2026-06-30-c2-auto-merge", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-30T00:00:00Z" + }, + "C3-auto-merge": { + "state": "promoted", + "ratified_by_change": "change/2026-07-01-c3-auto-merge", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-07-01T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" + } + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-blocking-checkers.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-blocking-checkers.json new file mode 100644 index 0000000000..34116dcb53 --- /dev/null +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/auto-merge-blocking-checkers.json @@ -0,0 +1,117 @@ +{ + "schema_version": "1.0", + "executor_class": "self-operated", + "dispatch_posture": "autonomous-enabled", + "isolation_bindings": { + "ci-pool-a": { + "L2": { + "substrate": "egress-denied-container-pool", + "substrate_class": "container", + "component_reachable_hosts": [ + "example.com" + ], + "probe_evidence": "probe-transcripts/ci-pool-a-l2.json", + "runtime_markers": { + "runner-pool": "ci-pool-a", + "region": "us-east" + } + } + } + }, + "merge_policy": { + "C1": "human", + "C2": "auto", + "C3": "auto", + "C4": "human", + "C5": "human" + }, + "verification_blocking": { + "deterministic": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + }, + "ai-review": { + "C1": "not-required", + "C2": "blocking", + "C3": "blocking", + "C4": "blocking", + "C5": "blocking" + } + }, + "escalation_routes": { + "gate-failure": "queue:escalations/gate-failure", + "verification-divergence": "queue:escalations/verification-divergence", + "admission-rejection": "queue:escalations/admission-rejection", + "demotion": "queue:escalations/demotion", + "structural-plan-approval": "queue:escalations/structural-plan-approval", + "untrusted-provenance": "queue:escalations/untrusted-provenance" + }, + "admission": { + "classification": { + "tracker-vcs-event": { + "autonomy:audit": "C1", + "autonomy:mechanical": "C2", + "autonomy:scoped": "C3", + "autonomy:structural": "C4", + "autonomy:untrusted": "C5" + } + }, + "rules": [ + { + "signal_class": "*", + "provenance": "*", + "work_class": "C1", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C2", + "disposition": "autonomous-eligible" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C3", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C4", + "disposition": "human-gated" + }, + { + "signal_class": "*", + "provenance": "*", + "work_class": "C5", + "disposition": "human-gated" + } + ], + "autonomous_concurrency": 1, + "items_per_run": 3 + }, + "promotion_state": { + "C2-auto-merge": { + "state": "promoted", + "ratified_by_change": "change/2026-06-30-c2-auto-merge", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-30T00:00:00Z" + }, + "C3-auto-merge": { + "state": "promoted", + "ratified_by_change": "change/2026-07-01-c3-auto-merge", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-07-01T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" + } + } +} diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json index 828e93f629..e1b422ade2 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3-late.json @@ -36,7 +36,7 @@ "ai-review": { "C1": "not-required", "C2": "not-required", - "C3": "advisory", + "C3": "blocking", "C4": "blocking", "C5": "blocking" } @@ -106,6 +106,12 @@ "ratified_by_change": "change/2026-07-20-c3-auto-merge", "evidence_window": "2026-06-01..2026-06-28", "ratified_at": "2026-07-20T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" } } } diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json index 2cbaef1627..9e75572668 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2c3.json @@ -36,7 +36,7 @@ "ai-review": { "C1": "not-required", "C2": "not-required", - "C3": "advisory", + "C3": "blocking", "C4": "blocking", "C5": "blocking" } @@ -106,6 +106,12 @@ "ratified_by_change": "change/2026-07-01-c3-auto-merge", "evidence_window": "2026-06-01..2026-06-28", "ratified_at": "2026-07-01T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" } } } diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json index b6bdd5970a..0a0959898c 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c2reearned-c3.json @@ -36,7 +36,7 @@ "ai-review": { "C1": "not-required", "C2": "not-required", - "C3": "advisory", + "C3": "blocking", "C4": "blocking", "C5": "blocking" } @@ -106,6 +106,12 @@ "ratified_by_change": "change/2026-07-01-c3-auto-merge", "evidence_window": "2026-06-01..2026-06-28", "ratified_at": "2026-07-01T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" } } } diff --git a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json index c7187566c1..29d320ab35 100644 --- a/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json +++ b/plugins/autonomy/skills/setup/evals/fixtures/security-binding/promoted-c3-only.json @@ -36,7 +36,7 @@ "ai-review": { "C1": "not-required", "C2": "not-required", - "C3": "advisory", + "C3": "blocking", "C4": "blocking", "C5": "blocking" } @@ -100,6 +100,12 @@ "ratified_by_change": "change/2026-07-01-c3-auto-merge", "evidence_window": "2026-06-01..2026-06-28", "ratified_at": "2026-07-01T00:00:00Z" + }, + "C3-ai-review-blocking": { + "state": "promoted", + "ratified_by_change": "change/2026-06-29-c3-ai-review-blocking", + "evidence_window": "2026-06-01..2026-06-28", + "ratified_at": "2026-06-29T00:00:00Z" } } } diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json index 8397af0594..44e41bc6dd 100644 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.manifest.json @@ -968,6 +968,19 @@ "exit": 0, "findings_substrings": [], "credential_roots": "/home/runner" + }, + "auto-merge-advisory-checkers.json": { + "exit": 1, + "findings_substrings": [ + "merge_policy.C2: \"auto\" with verification_blocking.ai-review.C2 \"advisory\"", + "merge_policy.C3: \"auto\" with verification_blocking.ai-review.C3 \"advisory\"" + ], + "credential_roots": "/home/runner" + }, + "auto-merge-blocking-checkers.json": { + "exit": 0, + "findings_substrings": [], + "credential_roots": "/home/runner" } }, "quarantined": {}, diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs index 6685d964cb..8d1798961e 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.mjs @@ -397,7 +397,14 @@ const PROMOTABLE_CELLS = new Set(["C2-auto-merge", "C3-auto-merge", "C3-ai-revie // (>= 20 autonomous C2 merges with 0 demotion events), so contrary evidence // against the prerequisite cell invalidates the dependent cell too — a failed // trust signal never leaves a promotion that was earned on it standing. -const PROMOTION_DEPENDENCIES = { "C3-auto-merge": ["C2-auto-merge"] }; +// APPEND, never prepend: `failed` is built in dependency-array order and a +// fixture pins the C2 cell as the named prerequisite, so reordering breaks that +// pin the moment both fail. C3 auto-merge's automatic transition is gated on the +// C3 AI-review cell being blocking — an advisory review layer gives a dissenting +// checker no force — so a demotion of that cell must lower auto-merge with it. +// Reached dynamically otherwise: the review cell resolves back to its advisory +// floor on contrary evidence while auto-merge stays effective-promoted. +const PROMOTION_DEPENDENCIES = { "C3-auto-merge": ["C2-auto-merge", "C3-ai-review-blocking"] }; // Admission shipped defaults per work class (the admission-policy leaf), and // the leaf's permissiveness order: autonomous-eligible > human-gated > @@ -1933,6 +1940,57 @@ function checkSemantics(binding, probeRoot, egressAllowList, credentialRoots) { } } + // Unanimity join. A class bound auto-merge takes its transition without a + // human, and the topology leaf's unanimity invariant requires every checker + // the class declares to agree first. The topology floor supplies the checker + // POPULATION; the security-review knob supplies its FORCE — unanimity needs + // both, and only force is configurable into absence, because floors are + // tighten-only. Two ways a binding removes it: an advisory layer, where the + // checker dissents and the transition proceeds anyway; and a not-required + // layer with model-adjudicated checkers DECLARED for the class, where the + // layer they judge in never runs, so their agreement can never be obtained. + // A class declaring no model-adjudicated checker is exempt from the second: + // its floor is seated by a deterministic slot, whose force is its own layer. + // Vendor-hosted is exempt entirely — the cap above already rejects every auto + // cell there, the same carve-out the promotion-ratification checks make. A + // missing or malformed knob already produced its structural finding; + // re-reporting it here would double-name one root cause. + if ( + isPlainObject(binding.merge_policy) && + isPlainObject(binding.verification_blocking) && + binding.executor_class !== "vendor-hosted" + ) { + for (const workClass of WORK_CLASSES) { + if (binding.merge_policy[workClass] !== "auto") continue; + for (const layer of LAYERS) { + const perClass = binding.verification_blocking[layer]; + if (!isPlainObject(perClass) || perClass[workClass] !== "advisory") continue; + findings.push( + `merge_policy.${workClass}: "auto" with verification_blocking.${layer}.${workClass} "advisory" — an automatic transition requires unanimous agreement among every checker the class declares, and an advisory layer records a checker's dissent without withholding the transition; set verification_blocking.${layer}.${workClass} to "blocking" (ratifying its promotion_state cell where the cell is promotable) or bind merge_policy.${workClass} to "human"`, + ); + } + const aiReview = binding.verification_blocking["ai-review"]; + if (!isPlainObject(aiReview) || aiReview[workClass] !== "not-required") continue; + const topology = isPlainObject(binding.verification_topology) + ? binding.verification_topology[workClass] + : null; + if (!isPlainObject(topology) || !Array.isArray(topology.roles)) continue; + const declaredModelCheckers = topology.roles + .filter( + (role) => + isPlainObject(role) && + CHECKER_TYPED_ROLES.has(role.role) && + isNonEmptyString(role.name) && + !isNonEmptyString(role.scanner_class), + ) + .map((role) => JSON.stringify(role.name)); + if (declaredModelCheckers.length === 0) continue; + findings.push( + `verification_topology.${workClass}.roles: model-adjudicated checkers [${declaredModelCheckers.join(", ")}] declared while merge_policy.${workClass} is "auto" and verification_blocking.ai-review.${workClass} is "not-required" — the layer those checkers judge in never runs, so their agreement can never be obtained and unanimity over them is vacuous while the merge proceeds automatically; set verification_blocking.ai-review.${workClass} to "blocking", drop the model-adjudicated roles and let the class's deterministic slot seat its floor, or bind merge_policy.${workClass} to "human"`, + ); + } + } + // Verification-topology floors and coverage per declared class. An absent // verification_topology (or an absent class within it) is NOT a hole: the // verification-topology leaf's shipped floors apply, exactly as From bfbba95b1eecd91d69bd3d19be934062966eb6b0 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 23:28:19 -0400 Subject: [PATCH 16/17] feat(autonomy): bind lens selection and the advisory narration lane on the operator surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two axes of verification policy do not belong on the security binding, and the reason is the asymmetry the split exists for. A floor the governed agents can lower is no floor — but the lens pool seats no slot and the narration lane has no binding cell at all, so neither can weaken one. What they can change is angle, never coverage: the reachable outcomes are a lensed checker or an unlensed one, never fewer checkers than the class's floor. That is what makes an operator surface the correct home rather than a convenient one. Lenses exist because a slot fixes WHO verifies and nothing yet fixed what they are asked to look for. Two checkers asked the identical question share the blind spot the count exists to cover — the same failure as two slots resolving to one model, one level up. The vocabulary is closed for the reason the rest of this contract is: a lens the pipeline cannot resolve to a question is a preference, and preferences are not policy. The narration lane cannot gate, and that is structural rather than a default. It has no cell on the security binding, none in the schema, and userConfig carries only whether it runs — so there is nothing an org could flip to promote it. A knob shipped off is a weaker property. The evidence supports the lane in this shape and no other. Model judgment over rendered artifacts tops out below gate precision, its characteristic error is calling broken things fine, and its verdict on identical input varies run to run in a way hosted inference gives the operator no way to tune away. Recall is where it is strong, which is what makes narration its job. It therefore ships declared and inert: its precondition is an upstream deterministic comparator this repository does not ship, and the contract says so rather than leaving a reader to discover it. The plugin manifest change is gated on a live fetch of the plugins reference (https://code.claude.com/docs/en/plugins-reference.md, fetched 2026-08-11), which also surfaced the constraint that matters for future consumers: substitution is REJECTED wherever a field runs in a shell, and rejection is a hard error rather than a fallback. These keys are hook-consumed, so a consumer reads them from the environment or exec form, never by substituting into a shell command. Version and CHANGELOG move here rather than at close-out, because the generated README options block and the changelog-parity check are both CI gates that fail the moment the keys land without them. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- plugins/autonomy/.claude-plugin/plugin.json | 18 +++- plugins/autonomy/CHANGELOG.md | 41 ++++++++ plugins/autonomy/README.md | 2 + .../guardrails/verification-topology.md | 96 +++++++++++++++++++ plugins/autonomy/skills/setup/SKILL.md | 19 +++- 5 files changed, 171 insertions(+), 5 deletions(-) diff --git a/plugins/autonomy/.claude-plugin/plugin.json b/plugins/autonomy/.claude-plugin/plugin.json index 5ef982bc13..e1856de5ed 100644 --- a/plugins/autonomy/.claude-plugin/plugin.json +++ b/plugins/autonomy/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "autonomy", - "version": "0.15.1", + "version": "0.16.0", "description": "Governed autonomous agent operation: role-topology, binding-seam, wiring-vs-advisor, telemetry, return-accounting, trigger-dispatch, per-work-class guardrail-matrix, standing-routine-catalog, and design-only runner-charter contracts for climbing the AI-adoption ladder, plus a guided-setup skill that discovers an adopting org's state, writes its schema-versioned binding, wires standards-pinned OTLP emission with a zero-cost file-artifact default, wires human-attested return capture at the task boundary, wires signal adapters with one governed dispatch entrypoint, binds the five-class guardrail matrix to an org's isolation substrates with an in-boundary live-validation probe before recording each fail-closed binding, and stands up standing-routine-catalog classes as scheduled temporal signal adapters behind the one governed queue with free scheduling defaults wired as reviewable changes and each routine's work-class mapping homed on the security surface.", "author": { "name": "Melodic Software", @@ -25,7 +25,7 @@ "lane_stop_gate_enabled": { "type": "boolean", "title": "lane-stop gate", - "description": "Opt an autonomous lane into the deterministic Stop-hook completion gate. Default OFF \u2014 a Stop-blocking hook must never engage for an interactive session. Honored from user or managed settings only (the gate reads those files itself); per-session lanes are armed by the claude-ops lane launcher instead. The env mirror is never authority (#1784).", + "description": "Opt an autonomous lane into the deterministic Stop-hook completion gate. Default OFF — a Stop-blocking hook must never engage for an interactive session. Honored from user or managed settings only (the gate reads those files itself); per-session lanes are armed by the claude-ops lane launcher instead. The env mirror is never authority (#1784).", "default": false }, "lane_stop_gate_sentinel": { @@ -43,7 +43,7 @@ "lane_stop_gate_arm_id": { "type": "string", "title": "lane-stop gate arm id (launcher-managed)", - "description": "Written by the lane launcher at launch: names this session's arm record in the plugin's own data directory (hooks/lane-stop-gate-arm.sh). A capability pointer, never authority by itself \u2014 the gate validates it, honors only a record in its install-derived store, and binds it to the first presenting session. Not set by hand.", + "description": "Written by the lane launcher at launch: names this session's arm record in the plugin's own data directory (hooks/lane-stop-gate-arm.sh). A capability pointer, never authority by itself — the gate validates it, honors only a record in its install-derived store, and binds it to the first presenting session. Not set by hand.", "default": "" }, "lane_notify_enabled": { @@ -63,6 +63,18 @@ "title": "lane-notify terminal channel", "description": "Audible bell + OSC 9 notification written to the controlling terminal for the lane-stop alert.", "default": true + }, + "verification_lens_pool": { + "type": "string", + "title": "verification lens pool", + "description": "Ordered, comma-separated pool of verification lenses the model-adjudicated checker slots draw from — one distinct lens per slot, in pool order. Tokens come from the closed vocabulary in the verification-topology contract leaf; an unrecognized token is recorded as unresolved and draws no lens, and a pool shorter than a class's model-adjudicated slot count leaves the remaining slots unlensed rather than repeating a lens. The pool contributes to no count: how many checkers a class runs, how they must differ, and whether one must be cross-vendor are floors on the org's security binding, outside this setting's reach.", + "default": "specification,adversarial,contract,regression,evidence" + }, + "visual_narration_enabled": { + "type": "boolean", + "title": "advisory visual narration lane", + "description": "Run the advisory visual narration lane: strictly downstream of deterministic detection, it writes a plain-language account of a difference the deterministic layer already found and attaches it to the run record for the human gate. Advisory only — it emits no verdict, fills no checker slot, is counted by no floor, and never gates a transition; no cell anywhere names it as authority. Default OFF: it is inert without an upstream deterministic comparator, and each narrated artifact is a metered vision-model call.", + "default": false } } } diff --git a/plugins/autonomy/CHANGELOG.md b/plugins/autonomy/CHANGELOG.md index 58d81d1b68..ba62c11d96 100644 --- a/plugins/autonomy/CHANGELOG.md +++ b/plugins/autonomy/CHANGELOG.md @@ -6,6 +6,47 @@ All notable changes to the `autonomy` plugin are documented here. Format follows Versions 0.1.0–0.7.0 predate this file (introduced with 0.7.1); their history lives in the merged work-package PRs (#333, #343, #356, #372, #377, #600, #676). +## [0.16.0] + +### Changed — ACTION REQUIRED for anyone with an existing `L2`/`L3` binding, or auto-merge bound + +- **Every `L2`/`L3` level binding now carries `component_reachable_hosts`, and a level without it + is UNPROVEN.** Target selection is what makes the egress assertion mean anything: a probe that + samples only hosts the surface's installed components never request certifies a boundary that is + in fact open. Measured, not theorized — 201,961 bytes of origin data crossed a global + default-deny through a component-installed allow rule. The field is the human-ratified set of + destinations those components may request, and the probe must cover it in FULL, since each + destination is a separate policy decision. **The empty list is a valid and meaningful value:** it + is the explicit claim that the surface installs nothing carrying policy rules of its own. **To + restore dispatch: ratify the list (or the empty list) on the level entry and re-probe so the + transcript covers it.** +- **A class bound `auto` merge with a verification layer below `blocking` is now an INVALID + binding.** An automatic transition requires unanimous agreement among the checkers the class + declares, and an advisory layer records a dissent without withholding the transition — so the + configuration promised a gate it could not deliver. Bindings that encoded this are rejected with + a finding naming the remedy. **To restore: set the layer to `blocking` (ratifying its promotion + cell where promotable), or bind the class to `human` merge.** Demotion now cascades from the + `C3` AI-review cell to `C3` auto-merge for the same reason. + +### Added + +- **Verification topology** (`reference/guardrails/verification-topology.md` + a sixth guardrail + matrix column): who verifies a change, how those verifiers must differ, and the per-class floor + for how many there are — expressed as pipeline roles, relational constraints, and predicates a + binding can actually evaluate, with no capability label anywhere in the contract. Floors ship as + `min_checkers` and `min_model_checkers` per class, both tighten-only on the agent-unwritable + security binding. `cross_vendor_required` is never vacuously satisfiable, and vendor disjointness + holds among the model-adjudicated slots rather than only against the generator. +- **`verification_topology`** as an optional top-level security-binding key modeling all three axes. + Absent is not a hole — the shipped floors apply, as `escalation_severity` already does — so + `schema_version` stays `"1.0"` and every existing binding keeps validating. +- **Two `userConfig` options:** `verification_lens_pool` (what angle each model-adjudicated checker + is asked to take) and `visual_narration_enabled` (an advisory narration lane, default off). Both + live on the operator surface rather than the security binding because neither counts anything — + the pool seats no slot and the lane has no binding cell at all, so neither can weaken a floor. + The lane is structurally incapable of gating: no cell exists anywhere through which authority + could be granted to it. + ## [0.15.1] ### Changed diff --git a/plugins/autonomy/README.md b/plugins/autonomy/README.md index 6c3dc66eb4..72a2af2b6d 100644 --- a/plugins/autonomy/README.md +++ b/plugins/autonomy/README.md @@ -193,6 +193,8 @@ reads it from. | `lane_notify_enabled` | boolean | `true` | `CLAUDE_PLUGIN_OPTION_LANE_NOTIFY_ENABLED` | Master switch for the operator alert fired when a lane stops without signaling completion. | | `lane_notify_os_toast_enabled` | boolean | `true` | `CLAUDE_PLUGIN_OPTION_LANE_NOTIFY_OS_TOAST_ENABLED` | OS-native desktop toast (macOS/Linux) for the lane-stop alert. | | `lane_notify_terminal_enabled` | boolean | `true` | `CLAUDE_PLUGIN_OPTION_LANE_NOTIFY_TERMINAL_ENABLED` | Audible bell + OSC 9 notification written to the controlling terminal for the lane-stop alert. | +| `verification_lens_pool` | string | `"specification,adversarial,contract,regression,evidence"` | `CLAUDE_PLUGIN_OPTION_VERIFICATION_LENS_POOL` | Ordered, comma-separated pool of verification lenses the model-adjudicated checker slots draw from — one distinct lens per slot, in pool order. Tokens come from the closed vocabulary in the verification-topology contract leaf; an unrecognized token is recorded as unresolved and draws no lens, and a pool shorter than a class's model-adjudicated slot count leaves the remaining slots unlensed rather than repeating a lens. The pool contributes to no count: how many checkers a class runs, how they must differ, and whether one must be cross-vendor are floors on the org's security binding, outside this setting's reach. | +| `visual_narration_enabled` | boolean | `false` | `CLAUDE_PLUGIN_OPTION_VISUAL_NARRATION_ENABLED` | Run the advisory visual narration lane: strictly downstream of deterministic detection, it writes a plain-language account of a difference the deterministic layer already found and attaches it to the run record for the human gate. Advisory only — it emits no verdict, fills no checker slot, is counted by no floor, and never gates a transition; no cell anywhere names it as authority. Default OFF: it is inert without an upstream deterministic comparator, and each narrated artifact is a metered vision-model call. | ### How to set these diff --git a/plugins/autonomy/reference/guardrails/verification-topology.md b/plugins/autonomy/reference/guardrails/verification-topology.md index 65a7614e8c..dbba144170 100644 --- a/plugins/autonomy/reference/guardrails/verification-topology.md +++ b/plugins/autonomy/reference/guardrails/verification-topology.md @@ -131,6 +131,102 @@ surface, outside the blast radius of the agents they govern; a floor those agent floor. An absent or invalid binding fail-closes to the shipped values above, which this leaf owns — the matrix cells are their glance restatement. +## Lenses + +A slot fixes WHO verifies; a lens fixes what that verifier is asked to look for. Diversity of lens +is the point — two checkers asked the identical question share the blind spot the count exists to +cover, exactly as two slots resolving to one model do. + +Lenses bind MODEL-ADJUDICATED slots only. A deterministic slot is not asked a question; its coverage +is fixed by its scanner class, and a lens on it would be decoration. + +The vocabulary is CLOSED — a lens the pipeline cannot resolve to a question is a preference, and +preferences are not policy. + +| Lens | The question the checker is asked | +|---|---| +| `specification` | does the artifact do what the item asked for | +| `adversarial` | how could it be defeated, abused, or driven to fail | +| `contract` | does it honor the declared interfaces and invariants of everything it touches | +| `regression` | what previously-working behavior does it disturb | +| `evidence` | are the claims made about the artifact supported by artifacts the run actually produced | + +**Draw rule.** Distinct model-adjudicated slots draw distinct lenses, in pool order. + +**The pool contributes to NO count.** Every floor above is counted over the slots the binding +declares, so no pool value seats a slot, unseats one, or substitutes for one. A pool shorter than a +class's model-adjudicated slot count leaves the remaining slots UNLENSED — judging the artifact +whole, which is what a checker did before this section existed — rather than repeating a lens. An +unrecognized token is recorded as unresolved and draws no lens, on the same footing as the Budget +ceiling: recorded, never enforcing. Angle is the only thing a pool can add, and the only thing it +can fail to add. + +The pool therefore binds in plugin `userConfig`, not on the security binding: nothing it can be set +to changes how many slots a class runs, how they must differ, or whether one must be cross-vendor. + +## Where each axis binds + +Two homes, and the difference is not convenience. An axis fixing HOW MUCH verification a class gets +binds on the org's security governance surface, outside the blast radius of the agents it governs. +An axis fixing WHAT ANGLE that verification takes binds on the operator's own plugin-option surface, +which resolves from user-scope, invocation-scope, and managed settings only — a watched +repository's in-tree settings are not read for plugin options, so a repo an agent can write cannot +dial its own verification. + +| Axis | Home | +|---|---| +| `min_checkers`, `min_model_checkers`, `cross_vendor_required` | security binding | +| slot distinctness, relational constraints, predicates | security binding, inside the class's declared slots | +| the lens pool | plugin `userConfig` | +| whether the advisory narration lane runs | plugin `userConfig` | + +**Raise, never lower.** The asymmetry is the whole reason the split exists. Tightening a binding +cell is legal; weakening one is invalid per the floor rule above. `userConfig` reaches no floor at +all and cannot be made to: the pool contributes to no count, and the narration lane has no cell to +weaken. A degenerate pool costs angle, never coverage — the slots still run, still resolve +distinctly, and still owe unanimity. The reachable outcomes are a lensed checker or an unlensed one, +never fewer checkers than the class's floor. + +## The advisory narration lane + +An OPTIONAL lane that reads a difference a deterministic layer has ALREADY detected and writes a +plain-language account of it into the run record. Advisory only: it emits no verdict, fills no +checker slot, is counted by no floor, and never gates a transition. Its output reaches the human +gate as narration attached to the deterministic finding it explains. + +**Its position is upstream-dependent, not configurable.** It never runs first, and never runs on an +artifact no deterministic layer flagged. A lane with nothing upstream of it has nothing to narrate +and produces nothing. + +**It carries no authority cell, by construction rather than by default.** The table above gives it +no cell on the security binding, its schema carries none, and `userConfig` carries only whether it +runs. Nothing an org could flip promotes it — a stronger property than a knob shipped off. + +**What this does NOT rule out, stated plainly.** A class may declare a model-adjudicated checker +SLOT whose `requires_modality` names an image input, and a security-review layer may gate on that +slot. That slot is a CHECKER: counted by the floors, held distinct, bound by every relational +constraint, and owing unanimity. It is a different governance object from this lane, which is +counted by nothing and owes nothing. The measurements below bear on both, and a class declaring such +a slot should read them — but only the lane is structurally incapable of gating. + +**Why the lane is shaped this way — measured, not assumed.** Model judgment over rendered UI +artifacts tops out below the precision a gate needs, and its characteristic error is declaring +broken things fine: the wrong direction for a check whose purpose is catching breakage. Its verdict +on identical input also varies run to run at a rate well above the level at which a suite stops +being believed, and that variance is a property of hosted inference the operator cannot tune away — +no temperature setting or seed removes it. Recall is where it is strong, which is precisely what +makes narration its job rather than judgment. Every established comparison product surveyed reaches +the same arrangement independently: detect the difference deterministically, narrate it with a +model, route acceptance to a person. Per-request image caps and per-frame metering make a lane's +artifact volume a cost bound the runner resolves against the bound instance's declared limits, never +a policy axis. The sourced measurements live in the pull request that introduced this lane and on +the issue it closes, deliberately outside the contract surface. + +**Not demonstrated at runtime.** With no runner built there is no runtime in which to exercise the +ordering, so the property claimed here is structural: no cell exists through which authority could +be granted. The runtime assertion — that a deterministic pass carrying a narration finding still +advances — is a DEFERRED item bound to the runner's build trigger, not a claim made here. + ## Two fixed invariants Neither is a knob, and no binding may relax either. diff --git a/plugins/autonomy/skills/setup/SKILL.md b/plugins/autonomy/skills/setup/SKILL.md index 2acb372a55..07578ff2d0 100644 --- a/plugins/autonomy/skills/setup/SKILL.md +++ b/plugins/autonomy/skills/setup/SKILL.md @@ -156,7 +156,8 @@ splits policy into. This section owns resolution; the [guardrail slice below](#g this order resolves. **Two-surface split.** Security-sensitive guardrail axes — isolation bindings with their -runtime markers, merge policy, verification blocking knobs, promotion state, escalation +runtime markers, merge policy, verification blocking knobs, per-class verification +topology, promotion state, escalation routes, admission rules and caps — bind ONLY in the security binding document in the settings-as-code home, outside the blast radius of the agents it governs. Its schema is contract-owned and ships at @@ -169,6 +170,16 @@ binding. Non-security axes remap in the additive `guardrails` section of the rep binding: class→label strings (which local label means which work class) and cost-tier→model names — vocabulary remaps only, never policy content. +**A third home that is not a governance surface.** Two axes of the +[verification-topology leaf](${CLAUDE_PLUGIN_ROOT}/reference/guardrails/verification-topology.md) +bind in this plugin's `userConfig` rather than either governance surface: the verification lens +pool, and whether the advisory visual narration lane runs. Neither counts anything — the pool +contributes to no floor, and the narration lane has no security-binding cell at all — so neither can +weaken a floor, which is what lets them sit on an operator surface. Plugin options resolve from +user, `--settings`, and managed settings only; a watched repository's own `.claude/settings.json` is +not read for them, so a repo cannot dial its own verification. The slice proposes neither as a +binding field. + **Layered resolution order.** Org-policy-home defaults → settings-as-code per-repo security binding → repo-local non-security remaps. Later layers refine earlier ones for NON-SECURITY axes only; no repo-local (agent-writable) value ever supplies or overrides a @@ -264,11 +275,15 @@ depends on the binding until that human-landed change exists. `isolation_bindings` (surface id → level token → substrate instance + the human-ratified `substrate_class` and `component_reachable_hosts` + `probe_evidence` + the non-forgeable `runtime_markers` the dispatch seam attests against), plus the merge policy, - verification-blocking knobs, escalation routes, and admission rules and caps — all on the + verification-blocking knobs, each class's `verification_topology`, escalation routes, and + admission rules and caps — all on the prepared security-binding change, validated by [`scripts/check-security-binding.mjs`](scripts/check-security-binding.mjs) against [`schemas/guardrails-security-binding.schema.json`](schemas/guardrails-security-binding.schema.json) before it is proposed. + The lens pool and the advisory visual narration lane are NOT binding fields — they resolve from + plugin `userConfig` per the [third home above](#guardrail-binding-resolution), and proposing + either here is invalid. 5. **Security-review wiring folds in here (no separate capability)** — the security-review policy is one part of this single guardrail slice, never a near-duplicate setup capability. Wire the [security-review leaf's](${CLAUDE_PLUGIN_ROOT}/reference/guardrails/security-review.md) two From 27a49d81c3735c02d9e795735f467bc6e0349e10 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Tue, 11 Aug 2026 23:30:16 -0400 Subject: [PATCH 17/17] docs(topics): prune the docker-sandbox-substrate contract slice The slice is Contract tier: committed on a task branch only, pruned before merge. Its durable outcome is the shipped contract set and this pull request's body; the evidence base and the per-phase designs live in the working directory the repository does not track. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0133WoxqBf7kLDviEfrcYL5Z --- docs/topics/docker-sandbox-substrate/PLAN.md | 838 ------------------ .../design/design-resolution.md | 42 - 2 files changed, 880 deletions(-) delete mode 100644 docs/topics/docker-sandbox-substrate/PLAN.md delete mode 100644 docs/topics/docker-sandbox-substrate/design/design-resolution.md diff --git a/docs/topics/docker-sandbox-substrate/PLAN.md b/docs/topics/docker-sandbox-substrate/PLAN.md deleted file mode 100644 index 3a7990e385..0000000000 --- a/docs/topics/docker-sandbox-substrate/PLAN.md +++ /dev/null @@ -1,838 +0,0 @@ -# docker-sandbox-substrate - -## Brief - -### TLDR - -Docker Sandboxes (`sbx`) was evaluated as an isolation substrate and as a possible route to an -agent-agnostic autonomous pipeline. It is a **substrate instance for a seam this repository already -specifies** — not a plugin, not a new repository, and not a framework. Integrating it costs **zero -files here**, because the guided-setup path already ships and the ladder forbids naming a product -instance in this repository at all. It was probed live on Windows 11 Pro under a hardened invocation -and **both isolation assertions failed inside the boundary**, which is the passing condition. - -The larger goal that emerged — running the full delivery lanes as separate autonomous runs — is -governed by one evidence finding that reshapes it: **verifier independence pays at the model level, -not merely the context level**, and **judge diversity outranks judge size**. Verification topology -becomes a configurable column on the existing guardrail matrix, keyed on roles and relations rather -than on capability labels, which do not survive a model release. - -### Goal - -Decide whether and how to adopt a kernel-separated local isolation substrate, and fix the shape of the -verification topology for an autonomous multi-run delivery pipeline — both grounded in current -authoritative sources rather than recall, and both expressed so that no vendor name enters this -repository's contract surface. - -### Constraints - -- **Instance names never enter this repository.** `guardrails/isolation-ladder.md` names substrate - CLASSES as marked examples only; the instance id has exactly one home — the `substrate` field in a - consumer's security binding, outside this repo. -- **Trust is earned by demonstrated boundary behavior**, never by licence, audit history, or version - number. The contract is deliberately silent on provenance and treats it as an org choice. -- **A policy the governed agents can lower is no policy.** Verification floors live on the - agent-unwritable security binding, as the per-item caps already do. -- **Independent aggregation, never deliberation.** Consensus-by-discussion measured worse than every - single-model baseline; this is a fixed invariant of the design, not a configurable knob. -- **The runner stays trigger-gated.** Nothing here fires a T4 build trigger; an off-the-shelf substrate - scales the platform wall rather than supplying the case for building past it. -- Fresh-docs mandate applies to every contract-surface change, per `CLAUDE.md`. - -### Acceptance criteria - -1. `Docker.sbx` is tracked in the dotfiles user-scope winget list, and `provisioning` is unchanged — - satisfied by `melodic-software/dotfiles#427`; the WHP optional feature proved unnecessary on a host - with Hyper-V already enabled. -2. A probe transcript exists showing **both** ladder assertions failing inside the boundary under - `--clone` + `deny-all`, with the outer context proving each target reachable/present first, and - `outer_context_networked: true` — satisfied by `.work/docker-sandbox-substrate/probe-evidence-sbx-l3.md`. -3. The isolation probe gains a **third assertion covering the workspace mount**, and its egress - assertion tests whether **data flows** rather than whether `connect()` fails. -4. Verification policy is expressed as roles + relational constraints + machine-checkable predicates, - with **no capability label** (`frontier`, `flagship`, `daily driver`) anywhere in it. -5. Per-class verification **floors** (minimum checker count, whether cross-vendor is required) live on - the security binding; lens selection and the advisory lane live in plugin `userConfig`. -6. The visual E2E lane is wired as **advisory only**, downstream of deterministic detection, and cannot - block. -7. Merge remains human-gated, with additional routing on implementer/checker disagreement; anything - auto-proceeding requires **unanimous** checker agreement. - -### Captured assumptions - -- `sbx` evidence is **version-bound** to `v0.38.0`. The product ships roughly every two weeks with an - actively changing security surface, so the probe result is a snapshot, not a standing property. -- The probe covered one kit (`shell`), one workspace, one host. The **default (non-clone) mode was not - probed** — Docker documents it as having no workspace isolation, so it is assumed to fail. -- `local-policy`'s `filesystem:read/write allow **` is assumed to govern what the host permits `sbx` to - mount, not what the guest can reach; the guest saw only the read-only workspace. Not probed directly. -- The pipeline-architecture evidence transfers from math/QA benchmarks to software delivery. The lane - flagged this as its central limitation; the software-specific evidence is thinner and newer. -- Most multi-agent papers do not run the compute-matched baseline (a five-stage pipeline versus one run - with five times the thinking budget), so measured gains are confounded by scaffold quality. - -### Out-of-scope - -- **A new repository or framework.** `sbx` scores agent-agnostic but is welded to Docker's own microVM - and is proprietary — a provider candidate, never a framework competitor. The prior peer-frameworks - verdict (compose, do not adopt wholesale) stands. -- **Building the runner.** Its design pack is complete and its build triggers are unfired. -- **Adopting an orchestrator now.** Neither agent-native option ships a gate, which is the part the - design most needs; the pipeline stays thin code against a provider seam so the choice stays - reversible. -- **Naming `sbx`, Multipass, Hyper-V, or any product in this repository's contract surface.** - -### Deferred questions - -- **Q20 — What exact form should the probe's workspace assertion take, and how should the egress - assertion be reworded to test data flow rather than `connect()` failure?** *(arbiter: - `/planning:plan`)* A connect-only probe grades this substrate wrongly: raw TCP `connect()` succeeds - because the interception layer accepts the SYN and then drops the session. -- **Q21 — Should the isolation ladder model kit-supplied allow rules?** *(arbiter: USER-RESERVED)* The - `shell` kit added `network allow openrouter.ai` on top of a global `deny-all`. Installing a kit can - widen egress; the ladder does not currently model this, and it is a supply-chain concern that - changes what a level binding certifies. -- **Q22 — Should the three genuinely-additive "software factory" concepts be recorded as deferred - items with triggers?** *(arbiter: USER-RESERVED)* Accepted in principle (Q11); the trigger wording is - unwritten. They are fleet-level economics as tracked output (partly covered by the telemetry and - return-accounting contracts), portfolio-scale multi-repo fan-out as a unit of work (absent — the - seams are per-run), and self-service golden paths for humans and agents (absent). -- **Q23 — What re-verification cadence applies to version-bound substrate evidence?** *(arbiter: - USER-RESERVED)* A ~2-week release cadence with three security fixes in three weeks means specific - flags, defaults, and guarantees rot; the durable findings are the architecture and the posture. - -## Plan - -### Goal - -**What**: harden the isolation probe so it measures data flow and the workspace mount, then express -verification topology as a configurable column on the existing guardrail matrix — floors on the -agent-unwritable security binding, lens selection and the advisory visual lane in plugin -`userConfig`, merge human-gated with disagreement routing. - -**Why**: the probe currently certifies `L2` while the whole host-execution attack class goes -unmeasured, and a connect-only egress test grades a microVM-class substrate wrongly. The pipeline -half of the Brief is decided but unbuilt: without a topology column, verifier count, model routing, -and lens diversity have nowhere to bind. - -### Standards grounding - -No standards index is present in this repository, so the ladder's absent-index inference applies. -Grounded on the ambient repository standards; nothing was fetched beyond them. - -| Surface | Sections cited | Layer provenance | -|---|---|---| -| Plugin contract surface | `CLAUDE.md` — Fresh-docs mandate; Design rules (repo-agnostic, configurable without editing the plugin, plugin-form-safe, versioned) | team | -| Branch / PR process | `CLAUDE.md` — Branching & PRs (`pr-issue-linkage` body contract); `AGENTS.md` — stage explicit paths | team | -| Contract-slice lifecycle | `docs/conventions/topic-docs/README.md` — slice pruned before merge (`contract-slice-prune-gate`) | team | -| Plugin acceptance | `docs/MIGRATION-PLAYBOOK.md`, `docs/PLUGIN-PHILOSOPHY.md` | team | - -Offer, not applied: persist a standards index so future plans resolve these surfaces deterministically -instead of inferring them. - -### Q20 — RESOLVED (this plan is its arbiter) - -Both halves resolve as a **rewording plus one additive assertion**, not a loosening of any existing -check. - -**Egress — test data flow, not `connect()`.** The probe becomes a TLS fetch whose passing condition is -that **the inner peer is not the origin, or no origin bytes were read**. Certificate verification alone -was the first draft and is NOT sufficient: an org that installs a TLS-inspection CA inside the boundary -makes the interceptor's certificate verify cleanly, which would grade a fully-intercepted boundary as -egress-capable in one direction and a legitimately-denied one as passing in the other. The -discriminator is therefore a **peer-certificate comparison against the outer context**, which needs no -trusted-CA assumption at all: - -- The outer probe records the peer certificate fingerprint of each reachable target. -- The inner probe records its own. Denial is proven when the handshake fails, or when the inner - fingerprint DIFFERS from the outer one (an interceptor, not the origin), with zero origin bytes read. -- `egress_denied.transport_outcome` ∈ `dns-unresolved` | `connect-failed` | `tls-failed` | - `peer-substituted` | `not-applicable`, one entry per probed host, positionally paired with `host`, - in the comma-separated form `credentials_absent` already uses. -- A completed handshake whose peer fingerprint MATCHES the outer one and from which application data - was read is the FAIL condition. It has no passing token, so it cannot be recorded as a pass. - -Three additional legs close gaps a single-target exit-code test leaves open: - -- **Client readiness.** A boundary with no working TLS client would otherwise "pass" trivially — a - missing tool is indistinguishable from a denied network. The same client must first succeed against - an in-boundary endpoint; a client that cannot be shown to run yields UNPROVEN, never a pass. -- **Multiple targets under different operators.** One denied destination does not establish - default-deny: a policy can allow specific hosts while denying the probed one. At least two DISTINCT - external targets are required, each independently proven reachable from the outer context. - - *Scope change, 2026-08-10, found at implementation.* ~~one of them a randomized name no allowlist - could have anticipated~~ — a randomized name cannot satisfy the recipe's existing outer-reachability - invariant (`outer_exit_code` must be `"0"`, because a target that fails everywhere "fails" inside too - and proves nothing). Unguessability and outer-reachability are mutually exclusive over DNS, and - outer-reachability is the older, load-bearing rule. Distinct operators is the nearest property that - survives it. -- **Address family recorded, never inferred.** IPv4 and IPv6 are probed and recorded separately; an - absent family is `not-applicable`, never counted as denied. - -**Workspace — host-write containment, proven from the OUTER side.** A third assertion named -`workspace_host_write_contained` — the longer name is deliberate, because the short form would -overclaim. It asserts that **no write performed inside the boundary becomes visible at the host -workspace path**, which is Q6's write-back scope, and it does NOT cover read exposure (see the named -limitation below). Verifying from outside is what makes one rule cover both substrate shapes: a -read-only mount fails the inner write, a clone-mode mount accepts it invisibly, and both are -contained. The inner exit code is deliberately NOT constrained — only host state is. - -Legs, in order: - -1. Outer, pre: snapshot the host workspace. Record `git_config_digest_pre` (the literal `absent` - where the workspace has no `.git/config`) and confirm every canary path is absent. -2. Inner: write a **randomized canary** at each of a small fixed set of shapes — a plain file, a - dotfile, and a `.git/`-relative path — and append to `.git/config`. Randomized names defeat a - substrate that special-cases a known probe path; the shape set covers case-folding and - hidden-file handling that a single literal path would miss. Record `inner_exit_code` for the - record only. -3. **Tear the boundary down, then** re-check. An immediate post-check can miss delayed propagation - on a caching or asynchronously-flushed mount, so the post-check runs after teardown, with paths - canonicalized on both sides before comparison. -4. Outer, post: every canary still absent (`host_post_absent: "0"`) and `git_config_digest_post` - equals `git_config_digest_pre`. - -`.git/config` is named explicitly because it is a command key ring — `core.fsmonitor` executes host -code on `git status` — so an assertion that only checked ordinary files would miss the documented -escalation path. - -**Where the host workspace path is not observable from the outer context** (a hosted ephemeral -executor whose storage the operator cannot inspect), the assertion records `not-applicable` and the -level is UNPROVEN for that surface. It never silently passes. - -**Named limitation, carried into the leaf, not hidden.** This assertion does not measure READ -exposure, and clone mode leaves reads fully open — so exfiltration of workspace contents is -unaffected by a passing result. Q6 scoped the assertion to write-back; widening it to reads is -recorded as a deferred item with its own trigger rather than implied by the assertion's name. - -### Approach - -Phase order is dependency-driven: the vocabulary leaf (Phase 2) must exist before any surface can -cite it, and Phase 1 is independent of every other phase. - -#### Phase 1: Probe hardening — data-flow egress + workspace host-write containment [DONE] - -**SHIPPED.** PR #2150 merged as `c1b4c629`; `autonomy` is at `0.15.0` on `main`. 34/34 checks, -11/11 review threads resolved, 474 fixture checks, 0 pre-existing pinned reasons changed. - -**Merge gate satisfied 2026-08-10.** Evidence: `.work/docker-sandbox-substrate/probe-evidence-hardened-recipe.md`. -All three assertions failed inside a live boundary. - -**Carried into Phase 2 — Phase 1 does NOT satisfy the Q21 probe obligation.** Target selection is -load-bearing, and the shipped recipe does not yet require a target drawn from what the run's installed -components may reach. Measured consequence: 201,961 bytes of origin data crossed a global -default-deny through a component-installed allow rule. Until that leg lands, a conforming transcript -can still certify a boundary that is open. - -**What the live run changed.** It was not a formality — it found two recipe defects and one stale -environment claim that no amount of review would have caught: - -- **The peer-identity design is empirically vindicated, not merely reasoned.** The measured boundary - presented a certificate with the CORRECT hostname signed by a CA it trusted, so - `ssl_verify_result=0` — certificate verification returned SUCCESS on a fully sealed boundary. Only - the differing fingerprint distinguished interception from reached egress. This substrate is a live - instance of the TLS-inspection case, not a hypothetical one. -- **A direct-TLS fingerprint tool cannot traverse an HTTP `CONNECT` proxy** and reports no peer at - all, identically for a sealed and an open boundary. The recipe now requires a proxy-aware capture. -- **The probe shape now shows fail-on-HTTP-error explicitly.** The first attempt at the live run - reproduced the original false negative exactly — a block page is a successful transfer. -- **Handoff correction, itself corrected during cleanup.** The global deny-all posture did NOT appear - in `policy ls` while two kit-scoped policies were present, which is what made it read as absent — and - the first version of this note wrongly concluded that the listing never displays the global policy. - Once both sandboxes were removed, the listing showed `default-deny-all` plainly. The mechanism behind - the earlier omission is unexplained and is NOT claimed here. What holds either way: - **`policy check network ` is the authoritative confirmation route**, because it reports the - effective decision rather than the listing's view of it. - -**PR review outcome (#2150).** Two reviewers — one cross-vendor, one same-vendor — independently -converged on the SAME five findings, which is what marked them real rather than stylistic. All five -were confirmed and fixed; 474 checks pass, 0 pre-existing pinned reasons changed. - -- **One was a stale-justification failure worth carrying forward as a lesson.** The plan recorded that - the non-zero-exit invariant "stands unchanged — no relaxation", justified by the claim that - certificate verification achieved the same discrimination while keeping the exit requirement. - Certificate verification was then REPLACED by peer-identity comparison, which destroyed that - justification, but the exit requirement survived unexamined — leaving a sealed boundary whose block - page carries a successful HTTP status unprovable. A zero exit is now accepted only where the entry's - outcome is `peer-substituted` and its fingerprints differ. **When a decision's premise is replaced, - re-derive every rule that premise was carrying.** -- **The other four shared one shape: counting entries where coverage was meant.** Two egress entries - counted as two targets (`example.com,example.com` passed); three canary strings counted as three - shapes (`same,same,same` passed); fingerprints went unchecked on outcomes that assert no peer; - `",,"` counted as three recorded exit codes. **Phase 3 inherits this directly** — a role list, a - relational constraint, and a predicate set are all count-shaped surfaces, and the same defect is - available in each unless the checks assert distinctness and coverage rather than length. -- **Pushed back on one half-suggestion rather than complying:** the checker has no trustworthy way to - verify two hosts belong to different operators, since every signal it could read comes from the same - transcript an executing agent authored. The recipe states the requirement; the checker enforces the - verifiable part; the gap is recorded rather than implied. - -Review: security - -Criterion 3. Delivers the Q20 resolution above. - -- [ ] **Pre-flight consumer check** (FIRST work item — this migrates the probe-transcript contract): - `Grep` for `egress_denied`, `credentials_absent`, `outer_context_networked`, and - `verifyProbeTranscript` across `plugins/`, `scripts/`, and `.github/`; document every parse - path. Known consumers at plan time: `scripts/check-security-binding.mjs`, - `scripts/check-security-binding.fixtures.test.mjs`, the fixtures manifest, and - `skills/setup/SKILL.md`'s guardrail slice. -- [ ] **Hardcoded assertion COUNTS must move with the assertion set** — verified present at - `templates/isolation-probe.md:6` ("the SAME two assertions"), `:13` ("Two checks"), `:103` - ("both assertions"), and `schemas/guardrails-security-binding.schema.json:178` ("capture shape - with both assertions"). Leaving any of them stale makes the contract self-contradicting. -- [ ] `templates/isolation-probe.md` — reword the egress assertion to the peer-comparison form and add - its readiness, multi-target, and address-family legs; add the - `workspace_host_write_contained` assertion with its outer-first, post-teardown probe shape and - per-substrate wrapping note; extend the transcript capture shape with both new blocks. -- [ ] `reference/guardrails/isolation-ladder.md` — the `L2` level text names default-deny egress and - credential protection; add host-write containment as the third property a boundary must - demonstrate, and state the read-exposure limitation in the same breath. Classes only, no - instance names. -- [ ] `scripts/check-security-binding.mjs` — extend `verifyProbeTranscript`: validate - `egress_denied.transport_outcome` (closed token set, one entry per host, positionally paired) - and the `workspace_host_write_contained` block. **Both new checks run LAST in the function** — - `verifyProbeTranscript` returns the FIRST problem it finds, and all 58 `probe-evidence-*` - fixture cases pin a `findings_substrings` naming their own specific rejection reason (verified: - `records assertions.egress_denied.host "192.88.99.1"` and the like). A new check running - earlier would return the new reason instead and rewrite every one of them. -- [ ] Transcripts referenced by PASSING fixtures gain both new blocks — exactly 6 JSON files: - `ci-pool-a-l2.json`, `ci-pool-a-l2-as112-v6.json`, `ci-pool-a-l2-multi-host.json`, - `ci-pool-a-l2-id-rsa.json`, `ci-pool-a-l3.json`, `ci-pool-a-l3-hosted.json` - (`ci-pool-a-l1-2026-07-01.txt` is an `L1` reference and is not verified). -- [ ] New NEGATIVE transcripts + manifest cases: missing `workspace_host_write_contained`; a canary - visible on the host post-teardown; `git_config_digest` changed; missing `transport_outcome`; - count mismatch against `host`; a matching peer fingerprint with data read; a single-target - egress probe; an unproven TLS client. -- [ ] `skills/setup/SKILL.md` — the guardrail slice's probe narration gains the third assertion. -- [ ] **Staged activation — this raises the `L2` bar for surfaces already bound.** Every deployed `L2` - binding was probed under the two-assertion recipe, so its transcript has no third block; the new - check makes those levels UNPROVEN, and the ladder's fail-closed rule then BLOCKS autonomous - dispatch on that surface. `L3` inherits it. That is the correct security outcome and it is a - breaking migration for every existing adopter, so it ships staged, never silently: - (a) the release notes and `CHANGELOG` state the bar raise and what re-probing costs; - (b) the checker's UNPROVEN finding text names the third assertion as the cause and points at the - updated recipe, so an operator reads a remedy rather than a bare rejection; - (c) the plugin `version` bump is MINOR at minimum, and the CHANGELOG entry is written as a - migration note. In-repo fixtures are NOT the migration — they are only the test of it. -- [ ] **Merge gate — re-probe on a real substrate.** The reworded egress assertion has never been run; - the existing evidence was captured under the OLD recipe. The stopped `probe-l3` sandbox is still - on this machine, so re-probing is cheap. At least one real substrate must be re-probed under the - hardened recipe before this phase merges, and the transcript committed as updated evidence. - -**Sanity Check:** - -- `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with every pre-existing case's - `findings_substrings` unchanged (diff the manifest: only ADDED keys, no MODIFIED `findings_substrings` - on pre-existing cases). -- The template documents the assertion in both places that matter, asserted precisely rather than by - token count: `grep -c "Workspace host-write containment probe shape" templates/isolation-probe.md` - returns 1, and `grep -c "workspace_host_write_contained" templates/isolation-probe.md` returns ≥ 1 - (the transcript capture shape). - - *Correction, 2026-08-10, found at implementation.* The original `≥ 3` literal-token count was - arbitrary and failed against a template that documents the assertion correctly in prose. Padding the - document to satisfy the count would have been the wrong repair. -- `grep -rn "SAME two assertions\|Two checks\|both assertions\|two assertions" plugins/autonomy/skills/setup/` - returns empty — every hardcoded assertion COUNT moved with the assertion set. - - *Correction, 2026-08-10, found at implementation.* ~~`must fail to CONNECT`~~ was wrongly included - in this zero-match list. That phrase belongs to the CREDENTIAL assertion's metadata-endpoint clause, - where connection-level failure is still the correct requirement; only the EGRESS assertion changed. - Deleting it would have damaged sound contract text to satisfy a bad check. -- `grep -rn "sbx\|Docker Sandboxes\|Multipass\|Hyper-V" plugins/autonomy/` returns empty. -- A re-probe transcript exists under `.work/docker-sandbox-substrate/` recording the new - `transport_outcome` and `workspace_host_write_contained` blocks from a live run. - -#### Phase 2: Verification-topology contract leaf + matrix column [DONE] - -**SHIPPED** as `f7d96afc`, then repaired twice under independent audit (`0c75e0a9`, `8172ab67`). - -**Both repair rounds found the SAME defect shape the phase before it did — a count that does not -guarantee the coverage it exists for.** Round 1: `min_checkers` counted role TYPES, so a class -declaring `[A, A, B]` satisfied a floor of 3. Round 2, inside the repair itself: relations and -predicates bind only model-adjudicated slots, so three DETERMINISTIC slots met `C4`'s floor while -`cross_vendor_required` bound the empty set of model slots and was vacuously satisfied — a binding -valid with no model judge at all, against a matrix cell that mandates AI review. **Three occurrences -across two phases makes this the effort's signature failure, not an incident.** The generalization -worth carrying: whenever a rule counts things, ask what it would accept if every counted thing were -identical. - -Other findings the audit closed: distinctness stated as "implied", which a validator cannot act on -because a slot NAME says nothing about what it resolves to; a budget ceiling written as invalidating, -which would have made this leaf the single enforcing exception to the matrix's own out-of-scope -statement on cost; three pointers citing support no file carried; and a deliberation clause resting -on an uncited measurement — replaced by the argument that is actually analytic, with the measurement -and its confidence grade moved to issue #2110. - -#### Phase 2 (original brief) - -Review: architecture - -Criterion 4, and the vocabulary Phases 3–5 cite. Documentation only — no schema, no code. - -- [ ] `reference/guardrails/verification-topology.md` (CREATE) — the normative leaf. Fixes the Q17 - vocabulary and nothing else: - **roles** `generator` · `checker` · `cross_vendor_checker` · `ranker`; - **relations** `distinct_model_from` · `distinct_vendor_from` · `not_weaker_than` (present but - NOT defaulted — no cross-vendor capability ordering exists to evaluate it against); - **predicates** `min_context_tokens` · `requires_modality` · `requires_feature`; - **budget** cost ceilings; **pin** `pinned_model_id`, append-only, reproducibility of a recorded - measurement only. - **The leaf contains no capability label at all** — not even as a rejected example. Criterion 4 - says "no capability label anywhere in it", and a rejected-vocabulary section would still put the - words in the normative artifact. The leaf states only that capability labels are rejected as - policy vocabulary, because they name a different thing at each vendor and do not survive a model - release; the sourced per-label rationale lives in the PR body and issue #2110, which is where a - future reader tempted to reintroduce one will find it. - States the two fixed invariants: **independent aggregation, never deliberation**, and - **unanimous checker agreement for anything auto-proceeding**. -- [ ] `reference/guardrails.md` — add the **Verification topology** column to the matrix, its - one-line column definition, and a glance-layer routing row to the new leaf. Depth stays in the - leaf; the hub gains no prose beyond the row. - -**Sanity Check:** - -- `grep -rniE "frontier|flagship|daily driver" plugins/autonomy/reference/` returns EMPTY — criterion - 4's "anywhere in it" is a zero-match assertion, not a scoped one. -- `plugins/autonomy/reference/guardrails.md` matrix header row contains `Verification topology`, and - the glance-layer table contains a row pointing at `guardrails/verification-topology.md`. -- Every plugin-internal link in the new leaf resolves (`skill-reference-verify` hook passes). - -#### Phase 3: Per-class verification floors on the security binding [DONE] - -**SHIPPED** as `16a50974`. 522 checks / 148 fixtures, manifest diff purely additive (16 added, 0 -removed, 0 pre-existing `findings_substrings` modified — verified semantically, not by eye). - -**The floor table gained a fourth axis the brief did not name.** `min_model_checkers` -(`C1` 0 · `C2` 0 · `C3` 1 · `C4` 2 · `C5` 2) exists because a total count cannot express which KIND -of coverage is owed. Without it the `C4` all-deterministic binding above is valid. -`cross_vendor_required` is now never vacuously satisfiable, and vendor disjointness holds among the -model slots rather than only against the generator. - -**One contradiction surfaced only by running it:** the pairwise-distinctness check demanded a -relational constraint between every checker pair, while the deterministic/model split rejects those -same constraints on deterministic slots — so a conforming binding was unrepresentable. Scoped to -model slots, since a cross-kind pair is distinct by construction. **A rule pair can be individually -sound and jointly unsatisfiable; only executing it shows that.** - -#### Phase 3 (original brief) - -Review: security - -Criterion 5, binding half. - -- [ ] **Pre-flight consumer check** (FIRST work item): `Grep` for `verification_blocking`, - `merge_policy`, and `schema_version` parse sites across the plugin and repo scripts. -- [ ] `schemas/guardrails-security-binding.schema.json` — add `verification_topology` as an - OPTIONAL top-level key. **Criterion 4 says the policy is expressed as roles + relations + - MACHINE-CHECKABLE predicates; a predicate that never reaches the schema is not machine-checkable, - so all three axes are modeled here, not only the count.** Per class: `min_checkers` (integer ≥ 1), - `cross_vendor_required` (boolean), and a role list whose entries carry the relational constraints - (`distinct_model_from`, `distinct_vendor_from`) and the predicates (`min_context_tokens`, - `requires_modality`, `requires_feature`). Optional-with-contract-defaults follows the - `escalation_severity` precedent, so `schema_version` stays `"1.0"` and all 113 existing fixtures - continue to validate. Absent binding is NOT a hole: the leaf's shipped floors apply, exactly as - `escalation_severity` falls back to contract defaults. -- [ ] `scripts/check-security-binding.mjs` — floors are FLOORS: tightening legal, weakening below the - shipped default invalid, no `override_justification` escape (the same rule - `verification_blocking` already carries). A relational constraint naming a role absent from its - own class's role list is invalid — that check is what makes the predicate machine-checkable - rather than decorative. -- [ ] New fixtures: a valid tightened binding; a weakened `min_checkers`; a `cross_vendor_required` - dropped below its floor; a non-class key; a `distinct_vendor_from` pointing at an undeclared - role; an unknown predicate token. - -**Sanity Check:** - -- `grep -c '"schema_version": { "const": "1.0" }' plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` - returns 1 (the version did NOT move). -- `node scripts/check-security-binding.fixtures.test.mjs` exits 0; the 22 pre-existing exit-0 cases - still exit 0 unmodified. -- Every axis named in the leaf appears as a schema key: for each of `min_checkers`, - `cross_vendor_required`, `distinct_model_from`, `distinct_vendor_from`, `min_context_tokens`, - `requires_modality`, `requires_feature`, `grep -c` in the schema returns ≥ 1. -- The undeclared-role fixture exits 1 with its pinned finding — proof the relational constraint is - enforced, not merely declared. - -#### Phase 4: Plugin `userConfig` — lens selection and the advisory visual lane [TODO] - -Review: code-design - -Criterion 5 (`userConfig` half) and criterion 6. - -- [ ] **Fresh-docs gate** (FIRST work item, non-negotiable — this is a plugin-manifest contract - change): open `docs/OFFICIAL-DOCS.md`, WebFetch - for the current `userConfig` schema and - `${user_config.KEY}` substitution rules, and cite the URL in the commit. No key is written - before that fetch. -- [ ] `.claude-plugin/plugin.json` — additive keys for lens selection and the advisory lane, each with - an explicit `default` (the plugin's existing keys all carry one). -- [ ] The visual lane is wired so it **has no blocking knob at all** — structurally incapable of - gating rather than defaulted-off-and-promotable. It is placed downstream of deterministic - detection and narrates what the deterministic layer already found. -- [ ] `reference/guardrails/verification-topology.md` — document the split: floors on the binding, - lens selection and the advisory lane in `userConfig`. Agents may raise their own verification, - never lower it. -- [ ] `skills/setup/SKILL.md` + `CHANGELOG.md` + `version` bump. - -**Honest limit.** Criterion 6 says the lane "cannot block". With no runner built, there is no runtime -in which to demonstrate that, and building one is barred by the Brief's own trigger-gate constraint. -So the criterion is met by **structural impossibility rather than a runtime test**: the lane is given -no blocking knob in `userConfig` and no `VerificationKnob` cell in the schema, so there is nothing an -org could flip. The runtime ordering assertion — deterministic pass plus visual fail still advances — -is recorded as a deferred item bound to the runner's build trigger, not claimed here. - -**Sanity Check:** - -- The visual lane's section in `reference/guardrails/verification-topology.md` contains no - `blocking` token: `grep -c blocking` over that section returns 0. -- `grep -c "visual" plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` - returns 0 — the lane has no binding-side knob at all, which is what makes it unpromotable. -- `node -e "JSON.parse(require('fs').readFileSync('plugins/autonomy/.claude-plugin/plugin.json'))"` - exits 0 and every new `userConfig` entry has a `default`. -- The commit message cites the fetched docs URL. - -#### Phase 5: Merge gating and disagreement routing [TODO] - -Review: security - -Criterion 7. - -**Routing question — RESOLVED, not deferred.** Verified this session: `verification-divergence` is -already a REQUIRED key in `escalation_routes`, and `guardrails.md` defines it as "a verification -outcome diverges from the expected or claimed result" — which is exactly implementer/checker -disagreement. Phase 5 binds to it and invents no token. The new machinery is the **unanimity -predicate**, not the event class. - -- [ ] `reference/guardrails.md` — merge-policy column keeps human merge; record that **every automatic - transition**, not merely a merge, requires unanimous checker agreement. Scoping it to merge alone - would leave an auto-advancing pipeline stage ungoverned, which is the hole criterion 7 exists to - close. -- [ ] `reference/guardrails/work-classes.md` — the unanimity requirement joins the promotion - discipline: a promoted `C2`/`C3` auto-merge cell still does not auto-proceed on checker dissent. - Promotion does not survive dissent; it is a ceiling, and dissent lowers the effective state the - same way contrary evidence already does. -- [ ] `scripts/check-security-binding.mjs` + fixtures — a binding whose `merge_policy` is `auto` for a - class whose `verification_topology` floor cannot express unanimity is invalid. - -**Honest limit.** Criterion 7's "unanimous agreement to auto-proceed" is a RUNTIME aggregation rule, -and the runner that would aggregate is design-only and trigger-gated — `runner.md` states "no build -begins until a T4 build trigger fires". This phase therefore delivers what is checkable without a -runtime: the contract obligation, the promotion-discipline consequence, and a binding-validity check -that a class cannot be configured to auto-proceed without a floor capable of expressing unanimity. -The per-run verdict-aggregation gate (unanimous pass, single dissent, checker timeout, duplicate -checker identity) is specified as a runner-seam obligation and lands with the runner build. The plan -does not claim enforcement it cannot demonstrate. - -**Sanity Check:** - -- `grep -n "unanimous" plugins/autonomy/reference/guardrails.md plugins/autonomy/reference/guardrails/work-classes.md` - returns a match in both — a documentation assertion, and labeled as such. -- The BEHAVIOR assertion: a new fixture binding `merge_policy.C3 = "auto"` with a - `verification_topology` floor that cannot express unanimity exits 1 with its pinned finding, and the - otherwise-identical binding with a conforming floor exits 0. -- `node scripts/check-security-binding.fixtures.test.mjs` exits 0 with the new dissent fixtures. - -#### Phase 6: Close-out [TODO] - -- [ ] Prune `docs/topics/docker-sandbox-substrate/` in the final commit before merge - (`contract-slice-prune-gate` is a required check — a PR landing the slice on `main` can never go - green). -- [ ] Paste the approved PLAN into the PR body inside a `
` block. -- [ ] Comment the outcome on issue #2110 and close the items it lists that this work discharges. - -**Sanity Check:** `git show --stat HEAD -- docs/topics/` shows only deletions; `gh pr view --json body` -contains the PLAN block, a closing keyword, and a non-empty `## Related` section. - -### Files Affected - -| File | Action | What changes | -|---|---|---| -| `plugins/autonomy/skills/setup/templates/isolation-probe.md` | Modify | Data-flow egress rewording; third assertion; transcript shape | -| `plugins/autonomy/reference/guardrails/isolation-ladder.md` | Modify | `L2` gains workspace containment as a demonstrated property | -| `plugins/autonomy/reference/guardrails/verification-topology.md` | Create | The Q17 vocabulary leaf and its two fixed invariants | -| `plugins/autonomy/reference/guardrails.md` | Modify | Verification-topology column, definition, routing row, unanimity note | -| `plugins/autonomy/reference/guardrails/work-classes.md` | Modify | Unanimity joins the promotion discipline | -| `plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` | Modify | Optional `verification_topology` floors | -| `plugins/autonomy/skills/setup/scripts/check-security-binding.mjs` | Modify | Transcript checks; floor checks; unanimity check | -| `.../evals/fixtures/security-binding/probe-transcripts/*.json` (6) | Modify | Both new assertion blocks | -| `.../evals/fixtures/security-binding/*.json` + manifest | Create | New negative and tightened-binding cases | -| `plugins/autonomy/.claude-plugin/plugin.json` | Modify | `userConfig` keys; `version` bump | -| `plugins/autonomy/skills/setup/SKILL.md` | Modify | Probe narration; topology setup | -| `plugins/autonomy/CHANGELOG.md` | Modify | Entries per phase-PR | - -### Alternatives Considered - -| Alternative | Why rejected | -|---|---| -| Bump `schema_version` to `2.0` for the new keys | Invalidates every adopting org's binding, which fail-closes their autonomous dispatch until re-authored. The schema's own `runner-*` and `escalation_severity` keys establish optional-additive as the house pattern | -| Make the third assertion optional-when-absent | A binding could keep certifying `L2` on two-assertion evidence — the silent degrade the ladder explicitly forbids. It rides the existing UNPROVEN path instead: the binding stays valid, the level stops counting toward eligibility | -| Assert workspace containment by requiring the inner write to FAIL | Grades clone-mode substrates wrongly — they legitimately accept the write and discard it. Verifying host state from the outer side covers both shapes with one rule | -| Certificate verification alone as the egress discriminator (this plan's own first draft) | An org that installs a TLS-inspection CA inside the boundary makes the interceptor verify cleanly, so the check grades an intercepted boundary as egress-capable. Replaced by a peer-fingerprint comparison against the outer context, which assumes no trusted CA | -| An origin-signed nonce challenge verified against an embedded public key | Strictly stronger, but no well-known public endpoint will sign a caller-supplied nonce, so it cannot be substrate-agnostic or vendor-neutral — it would require shipping and operating an endpoint, which this repository has no business doing | -| A new escalation mechanism for checker disagreement | `verification-divergence` already exists, is already required in `escalation_routes`, and its definition already means this — verified this session, so Phase 5 binds rather than deferring | -| Prohibiting automatic merge outright to satisfy criterion 7 | Overshoots the criterion, which permits auto-proceeding on unanimity, and would revoke the shipped `C2`/`C3` promotion path the guardrail matrix already grants. Unanimity is scoped to every automatic transition instead | -| Model the visual lane as a `security-review.md` layer with `blocking` defaulted off | A defaulted-off knob is promotable; measured 70% judge precision with a consistent over-crediting direction means it must never become a gate. No knob is the stronger form | - -### Test Strategy - -Test-first throughout — the fixture harness is already the red-green loop for this surface. - -- **Phase 1, 3, 5 (checker changes):** write the failing fixture FIRST (new negative transcript + - manifest case with its expected `findings_substrings`), confirm - `check-security-binding.fixtures.test.mjs` fails on it, then implement the check. This is the - established pattern for all 113 existing cases. -- **Regression floor:** the 22 pre-existing exit-0 fixtures must still exit 0, and no pre-existing - case's `findings_substrings` may change. That single assertion is what catches the check-ordering - hazard in Phase 1. -- **Phase 2 (docs only):** verification is the repo's own link and reference hooks plus the - vendor-name greps in the phase Sanity Check; no unit test applies. -- **Phase 4:** JSON parse plus a `default`-presence assertion on every new `userConfig` key; the - fresh-docs citation is verified by reading the commit message. -- **Phase 5:** the documentation greps are labeled as documentation assertions; the behavior assertion - is the paired fixture (auto-merge with a unanimity-incapable floor exits 1; the conforming twin - exits 0). -- **Covered only by a live re-probe, not by the fixture harness:** whether the reworded egress and - workspace assertions actually discriminate on a real substrate. Fixtures test the CHECKER, never the - RECIPE. This is why the live re-probe is a Phase 1 merge gate rather than a suggestion. -- **Not covered at all, and stated rather than hidden:** runtime behavior for criteria 6 and 7. No - runner exists to exercise them, and building one is barred by the Brief's trigger-gate constraint. - -### Risks and Mitigations - -| Risk | Likelihood | Impact | Mitigation | -|---|---|---|---| -| **Phase 1 blocks autonomous dispatch for every existing adopter.** Their `L2` transcripts predate the third assertion, so the levels go UNPROVEN and the ladder's fail-closed rule blocks the surface. `L3` inherits it | **Certain** | **High** | This is the intended security outcome of a bar raise, but it is breaking. Staged activation is a Phase 1 work item: remedy-bearing finding text, a migration CHANGELOG note, and a MINOR-at-minimum version bump. In-repo fixtures are the test of the migration, never the migration itself | -| The new transcript checks run before existing checks and rewrite 58 pinned `findings_substrings` | High | Med | Last-position ordering is a stated implementation constraint; the "no pre-existing substring changed" assertion catches it mechanically | -| A TLS-inspection CA trusted inside the boundary makes an interceptor's certificate verify cleanly | Med | High | Why the assertion is a peer-fingerprint COMPARISON against the outer context rather than plain certificate verification — it needs no trusted-CA assumption. First-draft cert-verification-only was rejected for exactly this | -| A boundary with no working TLS client "passes" trivially — a missing tool looks like a denied network | Med | High | The client-readiness leg: the same client must succeed against an in-boundary endpoint first, or the level is UNPROVEN | -| One denied target certifies default-deny while policy quietly allows others | Med | High | Two targets minimum, one a randomized name no allowlist anticipated. Note the adjacent kit-widening question is Q21 and stays USER-RESERVED — this leg strengthens the probe without deciding it | -| **The assertion does not cover READ exposure, and clone mode leaves reads fully open** | Certain | Med | Named limitation carried in the assertion's own name (`workspace_host_write_contained`) and stated in the leaf. Widening to reads is a deferred item with a trigger, not an implied guarantee | -| A caching or async mount propagates the inner write after the post-check | Med | High | Post-check runs after boundary teardown, with both sides canonicalized | -| Shipped floor VALUES are unevidenced | High | Med | OPEN DECISION 1 below — BLOCKS Phase 3, not the plan's approval | -| The hardened recipe is never re-run against a real substrate | Med | High | Promoted from advisory to a Phase 1 MERGE GATE — the stopped `probe-l3` sandbox is still on this machine | -| Criteria 6 and 7 are runtime claims with no runtime to test them in | Certain | Med | Both are met by structural impossibility plus contract obligation, and each phase states the limit explicitly. The runtime assertions are recorded as runner-seam obligations bound to the build trigger — not claimed as delivered | -| Pipeline evidence is pre-consensus and several findings are single-study | High | Med | The leaf records each choice's evidence basis, so a later contrary result demotes that choice explicitly rather than silently contradicting a rule with no stated warrant | -| Scope creep from Q21 into Phase 1 | Med | Med | Q21 is USER-RESERVED; no phase depends on it. If a phase starts to, that is drift and stops | - -### OPEN DECISIONS — not resolved by this plan - -1. **Shipped per-class floor values** (`min_checkers`, `cross_vendor_required`). **BLOCKS Phase 3**, - not this plan's approval. The interview fixed the SHAPE, never the numbers, and no research lane - sets them — so choosing them here would be a sizing guess dressed as evidence. RECOMMENDED starting - floors: `C1` 1/no · `C2` 1/no · `C3` 2/no · `C4` 3/yes · `C5` 3/yes. They are FLOORS, so an org may - only tighten, and the cross-vendor requirement lands on exactly the two classes whose cost the - evidence justifies. -2. **PR granularity.** RECOMMENDED: Phase 1 ships as its own PR — it is self-contained, - security-bearing, independently revertable, and it carries a breaking migration that deserves its - own release note. Phases 2–5 ship as a second PR carrying the topology change coherently. - Alternative: one PR per phase, five review round-trips. - -### Reserved decisions — RESOLVED 2026-08-11 - -All three were USER-RESERVED through planning and Phase 1. Research reframed two of them; the third -was settled empirically rather than argued. Evidence: -`.work/docker-sandbox-substrate/RESEARCH-reserved-questions.md`. - -#### Q21 — the ladder gains a class-level property, not a vendor-shaped rule - -**Decision: an `L2`+ binding must assert that nothing the run can install is able to WIDEN the -boundary — only narrow it.** - -The vendor already solves this under organization governance, where the documented precedence is -`kit allow ✗ / kit deny ✓` — *"Precedence is decided by a rule's decision rather than its source."* -It does NOT solve it in local-policy-only mode, which the vendor's table leaves unstated and which -this session measured: a component installed at sandbox-create time widened egress past a global -default-deny, and **201,961 bytes of origin data flowed**, with the origin's own CA on the wire. - -Naming the vendor's component type here would violate the ladder's classes-never-vendors rule, and -would not generalize. The property does: it covers browser-extension permissions, admission -controllers, and any additive policy engine. - -Work items (land with the Phases 2–5 PR, since Phase 1 already edits this file): - -- [ ] `reference/guardrails/isolation-ladder.md` — state the property on `L2`. An additive policy - engine whose components can only narrow satisfies it; one where an installed component can widen - does not, and that surface fails closed until governance is configured so it cannot. -- [ ] `templates/isolation-probe.md` — **the probe obligation this creates.** Target selection is - load-bearing: a probe sampling only hosts the installed components do NOT allow will certify a - boundary that is in fact open. The recipe must require probing in the configuration the run will - actually use, with at least one target drawn from what the installed components are permitted to - reach. -- [ ] Fixture: a transcript whose probed targets exclude every component-allowed host is not a - conforming capture. - -#### Q22 — the three software-factory gaps get triggers in the T4 idiom - -**Decision: drafted below in the runner charter's trigger idiom — a named, judgement-free condition, -explicitly not assumed to have fired.** Wording is for review. - -- **Fleet-level economics as tracked output.** *Trigger:* the return-accounting and telemetry contracts - are both bound and emitting for more than one repository under one org binding, AND a question is - asked of that data which per-run records cannot answer (cost or yield compared ACROSS repositories). - Until then the existing per-run contracts cover the need and a fleet aggregate would have no second - repository to aggregate. -- **Portfolio-scale multi-repo fan-out as a unit of work.** *Trigger:* a single work item requires - coordinated change across two or more repositories with a shared acceptance criterion, and the - per-run seams cannot express it without a human sequencing the runs. Until then every seam is - per-run by construction and fan-out has no unit to carry. -- **Self-service golden paths for humans and agents.** *Trigger:* a second adopter (any consumer - outside the authoring org) completes guided setup, OR the setup interview's unanswered-value rate - makes the interview itself the bottleneck. Until then a golden path would be generalized from a - single deployment, which is the sample size this repository already rejects elsewhere. - -Each is recorded as DEFERRED WITH A TRIGGER, never as rejected — the ladder's own "Rejected axis" -section is reserved for what was deliberately not chosen, which these are not. - -**Wording RATIFIED 2026-08-11 as drafted.** The three triggers stand verbatim; the review the user -reserved is closed. Nothing downstream depends on the phrasing, so a later revision costs a wording -commit and no rework. - -#### Q23 — event-triggered re-verification, with a staleness bound as backstop - -**Decision: re-verify on events that could change the probed property; cap evidence age separately. -No cadence keyed to release frequency.** - -The corpus already rejected the framing the question assumed. Release cadence is *evidentially inert*: -CISA warns against reading fix counts as a negative signal; Ozment & Schechter measure median -foundational vulnerability lifetime at **≥2.6 years** with ~67.6% found after 7.5; Rescorla cannot -exclude a constant discovery rate. Three weeks is not a sample, so a release-frequency cadence would -be ritual rather than control. - -- **Re-verification events:** a substrate version change touching the probed boundary; a policy-engine - or governance-mode change; a change to the installed component set (which Q21 just proved can widen - the boundary without any version change at all). -- **Staleness bound:** evidence older than the repository's existing **">2-month"** idiom is stale - regardless of events. Reusing that number rather than inventing one — it is already the corpus's - own gate. -- **What re-verification covers:** the specifics that rot — flags, defaults, guarantees. The - architecture and posture findings are durable and are not re-derived each time. - -### Deferred, with triggers — recorded so they are not silently implied - -- **Workspace READ-exposure assertion.** Trigger: any adopter binds a substrate whose workspace mount - is readable and whose threat model includes workspace exfiltration. Q6 scoped this round to - write-back; the assertion's name says so. -- **Runtime verdict-aggregation gate** (unanimous pass, single dissent, checker timeout, duplicate - checker identity). Trigger: the runner's T4 build trigger fires. Specified as a runner-seam - obligation in Phase 5; not deliverable before a runtime exists. -- **Runtime advisory-ordering assertion** (deterministic pass plus visual fail still advances). Same - trigger. - -These are distinct from Q21/Q22/Q23, which are USER-RESERVED and belong to the human, not to a -trigger. - -### Blast radius - -**HIGH.** The change touches an agent-unwritable security surface, a fail-closed checker with 113 -gated fixture cases, the isolation ladder's definition of `L2`, and a plugin manifest consumers -install. Triggers matched: security-sensitive surface; contract migration with downstream consumers; -fail-closed policy semantics. Phase 1 additionally halts autonomous dispatch for every existing -adopter until they re-probe, which is a breaking migration on a security floor. - -### Stress-test summary - -Two independent passes were attempted; one channel worked. - -- **Fresh-context sub-agent review (Step 3): FAILED TO DELIVER.** Three separate spawns each returned - an idle notification with no report. The subagent return channel is broken in this session. Recorded - rather than papered over, because Step 3 is mandatory and a silent skip would be the failure mode - the step exists to prevent. -- **Cross-vendor review (Codex): DELIVERED.** This is the route the skill names as PREFERRED over the - same-vendor sub-agent, so the fallback failing did not cost independence. Its findings drove the - revisions above: the TLS-inspection defeat of certificate verification, the client-readiness and - multi-target gaps, the async-propagation and canary-shape gaps in the workspace assertion, the - unaccounted `L2` migration, the documentation-only predicates, the capability labels leaking into - the normative leaf, and Phase 5 testing expressibility rather than behavior. -- **Findings verified before applying, not taken on trust.** Confirmed against the files: the - hardcoded assertion counts at `isolation-probe.md:6,13,103` and `schema:178`; criterion 4's literal - "anywhere in it"; `runner.md`'s "no build begins" (which is what makes criteria 6–7 runtime-untestable). - Confirmed harmless: `human-gated-only-no-l2.json` binds only `L1` and has an empty - `findings_substrings`, so the bar raise does not flip it. -- **Findings REJECTED with reasons:** prohibiting automatic merge outright (overshoots criterion 7 and - revokes a shipped promotion path) and the origin-signed-nonce challenge (cannot be vendor-neutral - without operating an endpoint). Both are recorded in Alternatives Considered. -- **What the stress-test does NOT establish.** It was an independent READING of the plan against the - files. Nothing was executed: no fixture harness run, no substrate re-probed, no checker exercised. - The review inherits the Test Strategy's own limit — it can find a wrong plan, not prove a right one. - The live re-probe merge gate exists because no amount of review substitutes for running the recipe. - -### Execution shape - -Phase 1 is file-disjoint from Phase 2 and depends on nothing; every other phase is gated. - -| Phase | Files | Overlaps with | -|---|---|---| -| 1 | probe template, isolation-ladder, checker, manifest, fixtures, transcripts, SKILL, CHANGELOG | 3, 5 (checker, manifest, fixtures) · 4 (SKILL, CHANGELOG) | -| 2 | guardrails.md, verification-topology.md (new) | 3, 4 (leaf) · 5 (guardrails.md) | -| 3 | schema, checker, manifest, fixtures, leaf | 1, 5 · 2, 4 | -| 4 | plugin.json, SKILL, leaf, CHANGELOG | 1 · 2, 3 | -| 5 | guardrails.md, work-classes.md, checker, fixtures | 1, 3 · 2 | - -Dependency graph: Phase 2 defines the vocabulary Phases 3–5 cite, so 2 gates all three. Phase 3's -schema is what Phase 5's binding-validity check reads, so 3 gates 5. **Phase 1 is independent of every -other phase.** - -**Recommended shape: sequential, 2 → 3 → 4 → 5, with Phase 1 free to run concurrently.** Phase 1 and -Phase 2 are genuinely file-disjoint, and under the recommended PR granularity they land in separate -PRs anyway — so concurrency there is free rather than orchestrated. Within the 2–5 chain the file -overlap on the checker and the topology leaf is heavy enough that parallelism would cost more in -conflict handling than it saves. - -| Phase | Surface | Basis | -|---|---|---| -| 1 | main-session | Security-bearing contract change with a breaking migration; judgment-heavy throughout | -| 2 | main-session | Normative contract prose; the vocabulary every later phase cites | -| 3 | main-session | Schema plus checker semantics on the agent-unwritable surface | -| 4 | main-session | Gated on a live docs fetch and a manifest contract change | -| 5 | main-session | Promotion-discipline semantics; the highest-consequence cell in the matrix | -| 6 | main-session | Close-out, prune, PR body, issue comment | - -No phase routes to a sub-agent worker. Two reasons, both real: every phase is judgment-heavy contract -work rather than mechanical volume, and the sub-agent return channel demonstrably failed three times -in this session. If a later session finds the channel healthy, Phase 1's fixture authoring is the one -slice that would delegate cleanly. - -### Decisions made (gate-passed) - -| Decision | What it changes in the plan | Basis (evidence) | -|---|---|---| -| `[EXEC-SHAPE]` Peer-fingerprint comparison, not certificate verification, as the egress discriminator | The Q20 egress resolution and the `transport_outcome` token set | Certificate verification is defeated by a TLS-inspection CA trusted inside the boundary; a comparison against the outer context's fingerprint assumes no trusted CA at all | -| `[EXEC-SHAPE]` The workspace assertion is named `workspace_host_write_contained` | The assertion name, the leaf text, and the deferred read-exposure item | Q6 scoped this round to write-back, and clone mode leaves reads fully open — the short name would have implied coverage the assertion does not provide | -| `[EXEC-SHAPE]` Post-check runs after boundary teardown, with randomized canaries across three path shapes | Phase 1's probe shape | A caching or asynchronously-flushed mount can propagate after an immediate check; a single literal path misses case-folding and hidden-file handling | -| `[EXEC-SHAPE]` Optional-additive schema keys; `schema_version` stays `"1.0"` | Phase 3's schema change and the 113-fixture regression floor | `escalation_severity` and the `runner-*` keys are the house precedent, described in the schema as preserving existing bindings. Verified: `const "1.0"` | -| `[EXEC-SHAPE]` New transcript checks run LAST in `verifyProbeTranscript` | Phase 1's implementation constraint and its Sanity Check | The function returns the first problem found, and all 58 `probe-evidence-*` cases pin their own rejection reason | -| `[EXEC-SHAPE]` The normative leaf contains no capability label at all, not even a rejected one | Phase 2's leaf content and its zero-match Sanity Check | Criterion 4's literal text is "no capability label ... anywhere in it"; the sourced rationale moves to the PR body and #2110 | -| `[FALLBACK — confirm or override]` Staged activation rather than an immediate hard cutover for the `L2` bar raise | A new Phase 1 work item and the top Risks row | The Brief did not anticipate that raising `L2` blocks dispatch for existing adopters. Fail-closed is correct; shipping it without a migration note is not | -| `[FALLBACK — confirm or override]` Criteria 6 and 7 are met by structural impossibility plus contract obligation, with the runtime assertions deferred to the runner build | The honest-limit notes in Phases 4 and 5, and two deferred items | `runner.md`: "no build begins until a T4 build trigger fires", and the Brief locks that constraint. The alternative would be claiming enforcement that cannot be demonstrated | -| `[EXEC-SHAPE]` Live re-probe promoted from advisory to a Phase 1 merge gate | Phase 1's merge gate and the Test Strategy | Fixtures test the checker, never the recipe; the reworded assertions have never been run against a real boundary | -| `[EXEC-SHAPE]` The Tier A design gate is satisfied by the interview register rather than re-running `/planning:design` | `design/design-resolution.md` exists instead of a design pack | Rounds 3–5 resolved every design thread and the register gated clean; each thread is mapped to its resolving question in that file | - -### Open questions - -- OPEN DECISION 1 (floor values) resolved 2026-08-11; Phase 3 shipped against it. -- Q21, Q22, Q23 were USER-RESERVED and are RESOLVED 2026-08-11 — see the resolutions below. Earlier - USER-RESERVED markers in this document predate that and are stale where they conflict. - -### Handoff to implementation - -#### User-approval gates - -- Both `[FALLBACK]` rows above, before the phase that implements them. -- OPEN DECISION 1, before Phase 3. -- Phase 1's merge gate: the live re-probe transcript is reviewed before the phase merges, because it - is the only evidence the reworded recipe works. - -#### Execution shape - -Sequential 2 → 3 → 4 → 5, Phase 1 concurrent and independent, all main-session. No scope-fencing -tables — no phase is delegated. - -#### Mechanical work - -Commit boundaries follow phases. Stage explicit paths, never `git add -A`. The contract slice -`docs/topics/docker-sandbox-substrate/` is pruned in the final commit before merge, and the PR body -carries the closing keyword plus a non-empty `## Related` section or CI fails on the linkage check. - diff --git a/docs/topics/docker-sandbox-substrate/design/design-resolution.md b/docs/topics/docker-sandbox-substrate/design/design-resolution.md deleted file mode 100644 index 5208fb9710..0000000000 --- a/docs/topics/docker-sandbox-substrate/design/design-resolution.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -outcome: early-exit -tier: A -resolved_by: /planning:interview rounds 3–5 ---- - -# Design resolution — docker-sandbox-substrate - -## Tier - -**A — design-significant.** Criteria 3–7 add contract vocabulary (a verification-topology axis set), -additive keys on an agent-unwritable schema, new checker predicates, and a new plugin `userConfig` -surface. That is a contract change across four components. - -## Why `/planning:design` is not re-run - -The design threads a Tier A gate exists to force were explored and RESOLVED by `/planning:interview` -rounds 3–5, whose register gated clean (`registered=23 open=0 deferred=4 blocked=0 withdrawn=5 -answered=14 brief=ok status=clean`). Each thread below names its resolving question; the register and -its evidence live in the topic's memory slice. - -| Design thread | Resolved by | Resolution | -|---|---|---| -| Where verification policy lives (module boundary) | Q16 | SPLIT — per-class floors on the agent-unwritable security binding; lens selection and the advisory lane in plugin `userConfig` | -| Verifier-policy vocabulary (the type surface) | Q17 | Roles + relational constraints + machine-checkable predicates (Axes A–E). Capability labels rejected with sourced reasons | -| Topology shape (new mechanism vs existing seam) | Q13 refined | A column on the existing guardrail matrix, not new machinery | -| Aggregation semantics | Q19 | Unanimous to auto-proceed; any dissent routes to a human. Deliberation barred as a fixed invariant | -| Escalation shape | Q18 | Fixed per-class floor plus disagreement-triggered escalation above it | -| Visual-lane placement | Q14 | Advisory only, downstream of deterministic detection | -| Orchestrator/framework selection | Q12 | None adopted; thin pipeline code against the existing provider seam | -| Probe assertion set | Q6 | A third assertion covering the workspace mount, generalizing to every substrate class | - -## What remains open at plan time - -- **Q20** — the exact FORM of the workspace assertion and the data-flow rewording of the egress - assertion. Arbiter is `/planning:plan`; resolved in the PLAN body, not here. -- **Q21 / Q22 / Q23** — USER-RESERVED. Parked; none of criteria 3–7 depends on them. - -## Override - -If the design threads above are judged insufficiently resolved, the correction is to run -`/planning:design` before implementation — not to widen the plan.