Repository navigation
Phase5 - #6
Phase5#6
Conversation
…I cards Wire the Need-to-Know release officer as a TrueForge agent: - src/agent/manifest.ts: buildAgentManifest() produces the POST /api/v1/agents payload with require_approval_for_tools: ["release_result"], dynamic subagents and Ask User Questions enabled, generative UI enabled, sandbox disabled. - src/agent/prompt.ts: root agent system prompt embedding the fixed authorized purpose/audience literals, the five-step tool flow, three parallel subagent roles, hard rules (no raw rows, no canary, release_result only after approved), and the three openui card templates (clearance/denial/receipt). - src/agent/subagents.ts: self-contained prompt constants for the planner, privacy-reviewer, and evidence-reviewer subagents. Each returns a typed JSON object and is explicitly forbidden from calling vault tools or approving releases. - src/agent/openui.ts: clearanceCard() and receiptCard() helpers that produce the canonical fenced openui blocks the model is prompted to emit. Purpose/audience tuple and both hashes are in the non-scrolling header region (EIP-7730 scroll-out mitigation); double quotes in values are escaped. - src/agent/agent.test.ts: 43 tests covering prompt invariants (authorized mission literals, approval gate, subagent names, canary absence, ask_user_question instruction), subagent prompt invariants (role field, column lists, tool prohibition), OpenUI card shape (fenced block, header labels, hash fields, quote escaping), and manifest shape (approval gate, enable_tools, feature flags, model params).
scripts/start-vault.ts: Opens the in-memory vault database, creates the store, and starts the Vault MCP server. Accepts --port or $VAULT_PORT (default 8788). Prints the listening URL and the TrueForge registration command on stdout. Handles SIGTERM/SIGINT with a clean shutdown (server close + db close). Uses the same Node 24 --experimental-strip-types loader as verify-receipt. package.json: Add "start-vault" script: node --experimental-strip-types --loader=./scripts/ts-loader.mjs scripts/start-vault.ts README.md: Replace the placeholder "instructions landing soon" with a concrete three-step getting started section: start the vault, register it and the agent in TrueForge, open the UI. Add verify-receipt usage. Update Status section to reflect all five phases complete.
PR Summary by QodoWire TrueForge agent flow and deterministic receipt verification
AI Description
Diagram
High-Level Assessment
Files changed (13)
|
Code Review by Qodo
1.
|
# Conflicts: # package.json
Gate A caught this live: the prompt told the model to call render_safe_chart before release_result, but the vault only serves charts for released aggregates, so every run hit a guaranteed not_released error — and the hard rules order the model to stop on any error. One run pushed through by disobeying its own stop rule; the next halted and asked the user, ending the turn without ever reaching the approval gate. The chart step now follows the release, with the vault's actual constraint stated beside it. AI-assisted (Claude Code); reviewed and understood by the author.
A VerifiableReceipt can now carry an evidence section naming the session and turns that produced it. Naming a session raises the bar: events become required, any event carrying session or turn ids must agree with the evidence, and some gated post-approval tool.response must witness the exact receipt — id and both hashes — because trueforge 0.1.4 responds to a denied gated call too, so mere execution-after-pause proves nothing. A denial leaves its error marker on the response and fails; a stream from another run cannot contain this receipt and fails as receipt_unwitnessed. The CLI fetches persisted events live when TRUEFORGE_BASE_URL is set: paginated at the server's 100-event cap, failing closed as events_partial when pagination cannot be proven honoured and events_unavailable on any fetch error. Events without a string type now fail closed as events_malformed instead of being skipped — skipped records would let evidence hide inside the stream unexamined. user.tool_approval events are not persisted by trueforge 0.1.4, so the explicit-grant rule applies when a log carries them and the structural rules above carry the proof when it does not. AI-assisted (Claude Code); reviewed and understood by the author.
Registers the ZAI model provider and the vault MCP server tolerating
409 as already-registered, then upserts the agent by id: list agents,
PUT /agents/{id} with {manifest} when the name exists, POST /agents
when absent. The agents API on trueforge 0.1.4 is id-routed —
name-based routes silently no-op. Base URLs use localhost throughout;
0.1.4 listens on IPv6 only and refuses 127.0.0.1.
AI-assisted (Claude Code); reviewed and understood by the author.
Gate A runs the full flow twice against a live TrueForge with the vault started in-process as ground truth. The deny path proves the pause happened and the vault recorded zero release transitions and zero receipts; the allow path proves exactly one receipt. Both sessions' persisted events are then refetched and the allow stream must witness the exact receipt — persisted approval events carry tool_calls ids only, no tool names, so gated calls are matched by id to their responses. The deny stream must carry the user-denial marker and no receipt. Writes the allow run as a VerifiableReceipt bundle with its session evidence. Gate B refetches the bundle's persisted events and checks the boundary: canary email and free text absent from every event byte, no sensitive column keys, customer ids, email addresses, or phone patterns in any vault JSON tool response (TrueForge's own openui syntax documentation is the one non-JSON tool response and carries a placeholder address that is not vault data), and verifyReceipt passes on the refetched events through the same path the CLI uses. Gate artifacts (gate-*.json) are run evidence, not source, and are excluded from Biome. AI-assisted (Claude Code); reviewed and understood by the author.
Phase 4 + Phase 5 completionThis PR now carries both phases end to end. Branch synced with main (all 14 PR #5 review fixes included via merge Phase 4 — TrueForge wiring: exit check MET, live
Phase 5 — evidence: exit check MET
Empirical findings worth knowing
Deliberate scope decisions (declines with reasons)
Gate artifacts ( |
AI-assisted (Claude Code); reviewed and understood by the author.
|
/review |
|
Code review by qodo was updated up to the latest commit 1ad24b5 |
There was a problem hiding this comment.
🔵 Needs a closer look
The evidence/witness verification and live-fetch security logic is safety-critical and non-trivial, and one latent boundary-scan issue plus a docs/port inconsistency warrant human review before approval.
Pull request overview
This PR (Phase 5) wires the Need-to-Know release officer into a TrueForge agent and completes the persisted-evidence verification path. It builds on the existing verify-receipt CLI by adding session/turn "evidence" binding: a bundle that names its TrueForge session must carry (or live-fetch) the persisted events that witness the receipt, and the verifier fails closed on partial fetches, mismatched sessions, or an unwitnessed receipt. It also adds the agent manifest/prompt/subagents/OpenUI card layer plus operational scripts (start-vault, setup-trueforge, gate-b).
Changes:
- Added
evidence(sessionId + turnIds) toVerifiableReceipt, with new outcomes (events_missing,session_mismatch,receipt_unwitnessed) and structural approval evidence for persisted streams that don't record the user's decision. - New paginated, fail-closed event fetcher (
events.ts) used by the CLI and gate scripts to verify against server-persisted events. - New agent wiring: root prompt, three advisory subagents, OpenUI card builders, agent manifest with
require_approval_for_tools: ["release_result"], plus setup/start/gate scripts and README docs.
File summaries
| File | Description |
|---|---|
| src/verify/verify.ts | Parses/validates evidence; enforces type-per-event, session match, and receipt witnessing; structural denial fallback |
| src/verify/receipt.ts | Adds evidence type, new outcomes, and VERIFIABLE_OPTIONAL_KEYS |
| src/verify/events.ts | Paginated fail-closed persisted-event fetcher |
| src/verify/events.test.ts | Tests pagination, offset-ignored, and unavailable paths |
| src/verify/verify.test.ts | Tests session-bound evidence, denial marker, unrecognised types |
| src/verify/cli.ts | Async main; live-fetches events when evidence + base URL present |
| src/agent/prompt.ts, subagents.ts, openui.ts, manifest.ts | Root/subagent prompts, card builders, and TrueForge manifest |
| src/agent/agent.test.ts | Invariant tests for prompts, cards, manifest |
| scripts/start-vault.ts, setup-trueforge.ts, gate-b.ts | Vault launcher, TrueForge registration, Gate B boundary checks |
| README.md, package.json, package-lock.json, biome.json | Run docs, scripts, SDK dep, ignore gate result files |
Review details
- Files reviewed: 17/19 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
escapeValue escaped quotes only, so a value ending in a backslash
neutralised its own closing quote and a newline started a fresh OpenUI
statement — injection, not formatting. Backslashes are escaped first,
then quotes, and control characters flatten to spaces since statements
are single-line. Regression test verified failing against the old
escape.
The CLI's live mode fetched only the turns the bundle claimed, so a
crafted bundle could omit the turn holding a denial or a canary leak
and still verify. The session's actual turn list is now fetched
(paginated at the 25-turn cap, same fail-closed rules) and must equal
the claimed list exactly; events are then fetched from the server's
list, never the bundle's. Verified live: the genuine Gate A bundle
passes, the same bundle with one turn omitted fails session_mismatch.
setup-trueforge's 409 path now says what it does: existing provider
and MCP configuration is kept, not updated — 0.1.4 settings routes
have no validated update path and name-based routes can silently
no-op, so rotation requires deleting the old entry first. The README's
unrunnable curl with a {...} placeholder is replaced by the
setup-trueforge command.
Found by Qodo review on PR #6.
AI-assisted (Claude Code); reviewed and understood by the author.
Verifier: a receipt now verifies only against the full vault chain — one root-thread prepare/validate/release call trio from the vault MCP server, arguments and responses cross-checked against the candidate, one approval gate resolving to the release call, one user allow between gate and response, and the released payload witnessed exactly. Evidence names the session, its inline agent, and a duplicate-free turn list; ids are format-validated and URL-encoded before any fetch, and event/turn pagination follows the server's page tokens, failing closed on overlap, missing metadata, or a turn list that changes under the fetch. Live mode authenticates the named session: it must exist, be owned by an inline (frozen) agent, and match the pinned manifest on the fields that matter — instructions, model, no seeded messages, the single gated vault server, subagents and sandbox disabled. Server-added defaults are ignored so a runtime default change cannot false-fail every bundle. Offline mode is an explicit CLI choice announced on stderr, never a silent downgrade. Agent: canary literals no longer appear in the prompt; release_result now carries the full approval tuple (purpose, audience, columns, suppressed cells, both hashes) which the vault revalidates at execution time (approval_tuple_mismatch); OpenUI cards use the real 0.1.4 grammar; dynamic subagents are removed and disabled — 0.1.4 children inherit the root's vault tools and would bypass the root-only release rule. Receipt ids are random UUIDs. setup-trueforge validates existing non-secret settings on 409 instead of silently keeping them. Proven live: Gate A and Gate B pass against the running TrueForge; verify-receipt passes in live and offline modes on the Gate A bundle, and a bundle with one turn omitted fails session_mismatch. 253 unit tests, tsc and biome clean. Gate artifacts are run evidence and stay untracked. AI-assisted (Claude Code); reviewed and understood by the author.
Phase 5's exit check asks for the enumerated negative-test count to be published: 89 verifier tests across verify, event-fetch, live-session, and boundary suites. AI-assisted (Claude Code); reviewed and understood by the author.
Seeds the Phase 9 limitations doc with the boundaries found during the completeness review: approval is structurally evidenced but not identity-bound (standalone 0.1.4 has no auth and no reason field), denied attempts are vault-audited but session-linked only through gate run artifacts until the Phase 9 published runs, and offline verification is announced as weaker than live. AI-assisted (Claude Code); reviewed and understood by the author.
|
/review |
|
Code review by qodo was updated up to the latest commit 5789a1e |
There was a problem hiding this comment.
🔵 Needs a closer look
The PR is large and touches security-critical fail-closed release verification whose correctness depends on subtle event-ordering and evidence-binding invariants that warrant final human review.
Review details
- Files reviewed: 34/36 changed files
- Comments generated: 3
- Review effort level: Balanced
Move gate-a-actions imports to the top of the file, give the malformed- candidate verdict a descriptive detail instead of echoing its outcome name, and correct the README: the frozen inline agent spec is checked once, while the turn list is the thing compared before and after the event refetch. AI-assisted (Claude Code); reviewed and understood by the author.
No description provided.