[MachineLearning-Nerd]

Integrates the TrueForge sandbox as a single post-release verification step, with the full gate suite re-proven live and five new consecutive clean runs published.

What it does

After a successful, human-approved release, the vault's render_safe_chart response now carries sandboxProof.canonicalPayloadBase64 — the exact canonical bytes behind the receipt's outputHash. The agent runs exactly one pinned sandbox command (printf '%s' '<b64>' | base64 --decode | sha256sum) and states that the recomputed digest equals the receipt's outputHash.

Gate A asserts the whole chain on persisted events: exactly one exec, positioned after the chart response, running the exact pinned command; exit code 0; the digest witnessed in the exec output; the digest independently recomputed by the gate from the persisted command bytes and required to equal the receipt outputHash; the model restating it afterwards; and zero sandbox activity in every session that never releases (deny, missing-purpose, exception).

Why the release path is unchanged

• Only already-released data enters the sandbox — the same rows the release response returned.
• The chart remains a deterministic in-vault renderer; no code execution sits anywhere pre-release.
• release_result stays the only approval-gated tool; sandbox execs pause nothing.
• The verifier's frozen-spec check now requires sandbox enabled (a spec with it disabled could not have run the required step, so drift fails closed).

Honest claims (docs updated)

• Sandbox use before a receipt is forbidden by the pinned prompt and detected by Gate A — not runtime-blocked.
• The standalone sandbox executes on the host: no hardened-isolation claim is made. The value is the independently witnessed recomputation, not isolation strength.

Evidence

• Unit suite 356/356 green; typecheck and lint clean.
• First live Gate A trial: all 34 checks PASS.
• Attempts 9–13: five consecutive CLEAN integrated runs, each with live verify-receipt PASS — published in docs/RUNS.md with session/turn/receipt IDs.
• Gates B and C and the reconnect proof re-passed live on the integrated build.

[MachineLearning-Nerd]

/review

[qodo-code-review]

PR Summary by Qodo

Verify released payload hashes in the post-release sandbox

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Exposes canonical released bytes for one post-release sandbox SHA-256 recomputation.
• Extends Gate A to verify the complete persisted-event proof chain.
• Documents security boundaries and publishes five consecutive clean integrated runs.

Diagram

sequenceDiagram
    actor Human
    participant Agent
    participant Vault
    participant Sandbox
    participant Events as Persisted Events
    participant GateA as Gate A
    participant Verifier
    Human->>Agent: Approve release
    Agent->>Vault: release_result
    Vault-->>Agent: Receipt outputHash
    Agent->>Vault: render_safe_chart
    Vault-->>Agent: Canonical bytes
    Agent->>Sandbox: Pinned SHA-256
    Sandbox-->>Agent: Recomputed digest
    Agent-->>Human: Confirm hash match
    Agent->>Events: Persist proof chain
    GateA->>Events: Validate sandbox proof
    Verifier->>Events: Check manifest and receipt

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Dedicated post-release verifier tool

• ➕ Enforces post-release-only execution at runtime
• ➕ Removes model-mediated base64 relay and shell construction
• ➖ Adds another trusted service or vault capability
• ➖ Does not exercise the standalone TrueForge sandbox seam
• ➖ Broadens the release architecture beyond this verification experiment

Recommendation: Keep the PR's pinned, single-command sandbox approach for the current operational-evidence goal: it minimally extends the released-data path, independently recomputes the digest in Gate A, and documents that pre-release misuse is detected rather than runtime-blocked. If sandbox discipline becomes a production security boundary, replace prompt-only restriction with a dedicated post-release verifier or runtime policy layer.

Files changed (14) +520 / -40

Enhancement (2) +171 / -0

gate-a-actions.tsValidate persisted sandbox hash proofs +164/-0

Validate persisted sandbox hash proofs

• Parses TrueForge system exec calls and independently hashes canonical payload bytes. Reports failures for command drift, ordering, activity count, mismatched output, missing successful responses, or missing assistant confirmation.

scripts/gate-a-actions.ts

handlers.tsExpose the released hash preimage +7/-0

Expose the released hash preimage

• Adds sandboxProof.canonicalPayloadBase64 to successful chart responses using only the canonicalized rows already released to the caller.

src/server/handlers.ts

Tests (5) +250 / -15

gate-a-actions.test.tsTest sandbox proof-chain gate helpers +178/-0

Test sandbox proof-chain gate helpers

• Adds positive and negative coverage for pinned commands, canonical bytes, event ordering, exit status, assistant restatement, exactly-one execution, and sandbox-free non-release sessions.

scripts/gate-a-actions.test.ts

gate-a.tsEnforce sandbox checks in live Gate A +22/-0

Enforce sandbox checks in live Gate A

• Runs the complete proof validation on the allow session and asserts zero sandbox creation or execution in deny, missing-purpose, and exception sessions.

scripts/gate-a.ts

agent.test.tsAssert sandbox-enabled agent policy +9/-2

Assert sandbox-enabled agent policy

• Updates manifest expectations and verifies that the prompt pins sandbox access to the exact post-release hash command and allowed material.

src/agent/agent.test.ts

handlers.test.tsVerify canonical sandbox proof bytes +22/-1

Verify canonical sandbox proof bytes

• Confirms the chart response encodes exactly the canonical released rows and that their SHA-256 equals the release receipt's output hash.

src/server/handlers.test.ts

live.test.tsReject sandbox-disabled frozen manifests +19/-12

Reject sandbox-disabled frozen manifests

• Refactors manifest-drift coverage into table-driven tests and proves that injected messages or disabled sandbox configuration fail live verification.

src/verify/live.test.ts

Documentation (4) +84 / -19

README.mdDescribe post-release sandbox verification +2/-2

Describe post-release sandbox verification

• Updates the architecture overview and live-gate guidance to include the post-release SHA-256 recomputation and its persisted-event validation.

README.md

LIMITATIONS.mdDocument sandbox scope and trust boundaries +19/-10

Document sandbox scope and trust boundaries

• Replaces the no-sandbox statement with the exact post-release use case. Clarifies host-local execution, model relay risks, and Gate A's independent command-byte verification.

docs/LIMITATIONS.md

RUNS.mdPublish five clean integrated sandbox runs +51/-3

Publish five clean integrated sandbox runs

• Raises the clean-run tally to 12 of 13 and records attempts 9–13 with session, turn, receipt, query, duration, and live-verifier evidence.

docs/RUNS.md

THREAT_MODEL.mdAdd sandbox to the threat model +12/-4

Add sandbox to the threat model

• Classifies sandbox output as untrusted, defines post-release-only discipline, and distinguishes persisted-event detection from runtime prevention and hardened isolation.

docs/THREAT_MODEL.md

Other (3) +15 / -6

manifest.tsEnable the TrueForge sandbox +5/-2

Enable the TrueForge sandbox

• Enables sandbox support in the generated agent manifest for the single post-release verification step while retaining deterministic in-vault chart rendering.

src/agent/manifest.ts

prompt.tsPin the post-release sandbox workflow +4/-0

Pin the post-release sandbox workflow

• Adds an eighth tool-flow step requiring one exact base64-decode and SHA-256 command after chart rendering. Hard rules prohibit every other sandbox command, pre-release use, and noncanonical input.

src/agent/prompt.ts

live.tsRequire sandbox-enabled live specifications +6/-4

Require sandbox-enabled live specifications

• Changes frozen-manifest validation to require sandbox enablement, ensuring a persisted specification could support the mandatory post-release verification step.

src/verify/live.ts

[Copilot]

🟢 Approval recommended

The sandbox proof material, pinned prompt/manifest expectations, and Gate A persisted-event assertions are consistent end-to-end and are backed by targeted unit tests for both the vault response and the gate logic.

Pull request overview

This PR integrates a single, pinned TrueForge sandbox exec as a post-release verification step to independently recompute the released payload’s SHA-256 from vault-supplied canonical bytes and prove it matches the release receipt’s outputHash, with Gate A enforcing the full evidence chain on persisted events.

Changes:

• Add sandboxProof.canonicalPayloadBase64 to render_safe_chart responses so the agent can recompute the release hash from canonical bytes.
• Enable sandbox in the agent manifest/pinned prompt and update live-verifier manifest expectations accordingly.
• Extend Gate A to (1) require the post-release sandbox hash proof in allow sessions and (2) require zero sandbox activity in non-release sessions; add/adjust tests and documentation.

File summaries

File	Description
src/verify/live.ts	Updates expected inline agent manifest checks to require sandbox enabled.
src/verify/live.test.ts	Refactors manifest rejection test cases; adds “disabled sandbox” rejection case.
src/server/handlers.ts	Adds sandboxProof.canonicalPayloadBase64 to chart responses from released rows.
src/server/handlers.test.ts	Verifies sandbox proof bytes canonicalize to released rows and hash to receipt outputHash.
src/agent/prompt.ts	Pins the exact single sandbox exec command and post-exec statement requirement.
src/agent/manifest.ts	Enables sandbox in the agent manifest and updates manifest documentation comments.
src/agent/agent.test.ts	Updates tests to assert sandbox enabled and prompt pins the sandbox step/command.
scripts/gate-a.ts	Wires in new sandbox proof and sandbox-activity assertions into Gate A.
scripts/gate-a-actions.ts	Implements persisted-event checks for the sandbox hash proof chain + no-sandbox-in-nonrelease sessions.
scripts/gate-a-actions.test.ts	Adds unit tests covering the sandbox proof chain and non-release sandbox activity detection.
README.md	Documents the post-release sandbox hash recomputation and Gate A’s persisted-event assertion.
docs/THREAT_MODEL.md	Updates trust boundaries and enforcement list to include sandbox as untrusted post-release compute.
docs/RUNS.md	Adds integrated-build run evidence and clarifies tally across pre-/post-sandbox attempts.
docs/LIMITATIONS.md	Updates limitations to describe post-release-only sandbox use and non-isolation claims.

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. Negated equality passes Gate A ✓ Resolved 🐞 Bug ≡ Correctness

Description

sandboxHashProofFailures accepts any later assistant message containing outputHash, so a
statement such as “sandbox digest does not equal <hash>” satisfies the gate. This allows a clean run
even though the model failed the required post-exec equality assertion.

Code

scripts/gate-a-actions.ts[R481-484]

+      index > execResponse &&
+      event.type === "model.message" &&
+      event.thread_id === "main" &&
+      assistantPresentation(event).includes(outputHash),

Relevance

●●● Strong

Accepted history flags false-positive gate predicates; negated equality is a direct correctness gap
in this new proof gate.

PR-#6
PR-#7

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new prompt requires the assistant to state that the sandbox-computed digest equals the receipt
hash, but the new gate predicate proves only that the later presentation contains the hash; the new
test covers absence of a statement but not a contradictory statement containing the digest.

src/agent/prompt.ts[45-47]
scripts/gate-a-actions.ts[479-487]
scripts/gate-a-actions.test.ts[1109-1119]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Gate A currently treats the receipt hash appearing anywhere in a later assistant message as proof that the assistant stated the sandbox digest equals the receipt `outputHash`. Negated, ambiguous, or unrelated text containing the hash therefore passes.

## Issue Context
The pinned agent prompt requires one plain-text sentence explicitly stating equality, while the gate is intended to prove that behavior from persisted events.

## Fix Focus Areas
- scripts/gate-a-actions.ts[479-487]
- scripts/gate-a-actions.test.ts[1109-1119]

Parse or strictly match the expected equality sentence rather than using a bare substring check. Add negative tests for negated and unrelated messages that contain the correct digest.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context sources

Review mode: 🧠 Deep: This behavior-changing security/evidence-path PR enables sandbox execution and adds substantial verifier/gate logic across multiple independent runtime and persistence paths, creating a high density of easy-to-miss defects.

Tip of the day

💡 Did you know, you can group findings by type and pick your Finding display, from Minimal to Full

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[Copilot]

🔵 Needs a closer look

It changes security-sensitive posture (enabling agent sandbox code execution, exposing released bytes, and altering the verifier's fail-closed spec check), which warrants final human review despite being well-tested and internally consistent.

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 1
• Review effort level: Balanced