Research: the attested-evaluation epic covers offline backtests only — decide whether the LIVE decision path needs attestation, and in what order

[JSONbored]

Problem

The attested-evaluation epic (#8534) attests the offline backtest replay — #8537: "run the backtest
replay inside the CoCo runtime class." Nothing in the epic attests the live gate decision path: the
merge/close calls that actually act on contributor PRs and drive upstream rewards.

apps/loopover-ui/content/docs/verify-this-review.mdx ~96 states the residual precisely and honestly:

Not proved: that the live gate ran this exact code when it made its decisions.

#8538 updates that passage for attested backtest runs. After the entire epic lands, the sentence
above remains true of every real decision. That is a defensible scope for the hosted-customer threat
model #8136 analysed — a customer wants to check a published statistic. It is the wrong scope for a
network evaluating whether our per-decision verdicts are honest, because for them the artifact under
scrutiny is the decision, not the aggregate.

Why this is now on the critical path

The trust question has been asked from outside. #8136's threat table calls this T3 — "the
maintainer's infrastructure is untrusted outright" — and concludes it is covered by neither
reproducibility nor attestation as currently scoped, because attestation proves computation, not
data provenance. #8540 holds the T3 research and is explicitly demand-triggered:

The trigger is the first hosted tenant (or an equivalent trust requirement from the network) asking
for provenance guarantees beyond attested execution.

That condition is now met. This issue exists so the scope gap is legible on the roadmap rather than
discovered when someone reads the epic and assumes it covers live decisions.

What "flawless" would actually require

Attestation of the live path is the expensive end. The cheaper mechanisms should be evaluated first, and
in this order — each is independently useful and none blocks on hardware:

1. Make the existing artifacts reachable and complete. The decision ledger's verify endpoint 401s in
   production (orb(trust): the 'public' decision-ledger verify endpoint requires an API token — 401 in prod, so the chain is verifiable by nobody #9120); six of seven actuation sites write no record at all (orb(trust): six of seven actuation sites write no decision record — cap, nag and approved-merge closes are invisible to the ledger and excluded from the certified guarantee #9134); the record is never
   published (orb(trust): the decision record is never published — contributors get a 12-char prefix of a document no endpoint returns, and 51 preimages are already destroyed #9123); its commitments miss the model, the effective prompt, and the calibrated threshold
   (orb(trust): the decision record's commitments don't commit — modelId is always null, promptDigest ignores ten prompt suffixes, configDigest misses the calibrated close floor #9124). None of this needs a TEE, and all of it is prerequisite — attested execution reading from an
   incomplete, unreachable ledger proves nothing.
2. External anchoring (orb(trust): the decision ledger has no external anchor and verify returns no hash — wholesale re-chaining and tail truncation both pass clean #9122). Without it the chain is self-attested and re-chainable by whoever
   holds the database. For a network verifier this is the single highest-value addition, and an on-chain
   commitment via the SN74 hotkey is a natural anchor because it lands somewhere neither party controls.
3. Then, and only then, attested execution of the live path — scoped here.

Requirements

• Written scope decision, in the Research: is TEE hardware attestation necessary, or does a reproducible-backtest checksum approach cover the real threat? #8136/Research: survey viable TEE/confidential-compute options on Cloudflare Workers/Containers #8137 two-spike format, recorded as a comment on this issue:
  what would it take to run the live gate decision inside the confidential runtime class, what does that
  cost (latency per PR, blast radius of an attestation failure, what happens when the KBS is
  unreachable mid-review), and is it worth it versus anchoring plus complete records?
• Explicitly answer whether a partial form is sufficient — e.g. attesting only the deterministic
  rule-evaluation stage while the model call stays outside the boundary, since the model call is already
  the recorded nondeterministic input under trust: deterministic replay harness with the model call as the sole recorded nondeterministic input #8838.
• If the answer is "not worth it": say so plainly, and make verify-this-review.mdx state permanently
  that live-path execution is out of scope, so the honest limit is documented rather than implied.
• Do not start implementation under this issue.

Links & Resources

• Epic: attested evaluation on the hosted bare-metal execution plane #8534 (parent epic — this is its scope boundary), orb: run the backtest replay inside the CoCo runtime class and persist its attestation envelope #8537, docs+ui: extend verify-this-review with the attested-run verification path #8538, Research: override-event provenance receipts (T3) — demand-triggered design #8540 (T3 research; trigger has fired),
  Research: is TEE hardware attestation necessary, or does a reproducible-backtest checksum approach cover the real threat? #8136 (threat table), Research: survey viable TEE/confidential-compute options on Cloudflare Workers/Containers #8137 (viability survey)
• Prerequisites: orb(trust): the 'public' decision-ledger verify endpoint requires an API token — 401 in prod, so the chain is verifiable by nobody #9120, orb(trust): the decision ledger has no external anchor and verify returns no hash — wholesale re-chaining and tail truncation both pass clean #9122, orb(trust): the decision record is never published — contributors get a 12-char prefix of a document no endpoint returns, and 51 preimages are already destroyed #9123, orb(trust): the decision record's commitments don't commit — modelId is always null, promptDigest ignores ten prompt suffixes, configDigest misses the calibrated close floor #9124, orb(trust): six of seven actuation sites write no decision record — cap, nag and approved-merge closes are invisible to the ledger and excluded from the certified guarantee #9134
• apps/loopover-ui/content/docs/verify-this-review.mdx ~89-100

Boundaries

Research and scope decision only. No implementation, no hardware commitment, no change to what any live
gate rule does.

maintainer-only — trust-architecture scope.

[linear-code]

JSO-6316

[JSONbored]

Scope decision (2026-07-26) — in the #8136/#8137 two-spike format, per this issue's own requirements.

1. What would attesting the live path cost?

Latency per PR. The live gate decision runs on every webhook (open/sync/review), not on a
scheduled batch like the offline backtest replay. Wrapping the deterministic rule-evaluation stage
in a CoCo/Kata confidential-VM boundary adds VM warm/launch latency, an attestation round trip to
the KBS to unseal secrets before the workload can read anything, and report generation/signing —
on top of a pipeline that already spends seconds on GitHub REST/GraphQL, D1, and the AI provider
call. A few hundred ms is probably tolerable. The KBS round trip is the real risk, below.

Blast radius of an attestation failure. This is the sharper cost, and it is categorically
different from the backtest case. A backtest run's attestation failing just means that run
publishes without the strongest evidence tier — the number still lands. A LIVE gate decision
failing attestation forces a choice: fail CLOSED (block every PR on this instance until the
runtime re-attests) or fail OPEN (proceed unattested, silently degrading the guarantee). Every
existing fail-safe posture in this codebase — the precision breakers, the close-audit holdout
instrument (#8831/#9135), the calibration writes — fails OPEN specifically so a measurement-layer
outage never takes down the review pipeline itself. Attestation would be the first place that
principle is tested for real: failing open makes the guarantee worthless on demand (wait for or
trigger a KBS blip), and failing closed turns a KBS/attestation-service outage into a full code-review
outage for every repo on the instance — a single point of failure the rest of the architecture has
deliberately avoided.

KBS-unreachable mid-review. The attestation flow needs the KBS reachable at least at launch to
unseal secrets, and a network partition to it (its own infra, its own outage modes) either stalls
in-flight reviews (an availability regression) or requires a cached/pre-fetched credential path that
partly defeats the point (secrets held outside the attested boundary during the outage). Neither is
clean, and both need new incident runbooks the offline backtest path never required — a stalled
batch job is a delayed metric; a stalled live gate is a contributor staring at "in progress"
indefinitely.

2. Is it worth it versus anchoring + complete records?

Against #8136's threat table (T1 tampered corpus, T2 wrong code ran, T3 infra untrusted outright),
the live path's real gap sits closer to T3: a network evaluator does not just want "the
deterministic rule engine ran the published code" (attestation) — they want "the recorded decision
honestly reflects what happened," which is a provenance question, and #8136 already concluded
attestation does not answer that alone ("attestation proves computation, not data provenance").

This same batch of work (#9120/#9122/#9123/#9124/#9134) makes the decision record complete,
reachable, and — via #9122 — externally anchored. That closes the gap a network verifier actually
has today: a self-attested, re-chainable-by-the-operator ledger nobody outside the instance can
check. Attested execution on top of an anchored, complete record would add "the deterministic stage
ran the real code" — genuinely separate value, but only reachable once the ledger itself is
trustworthy, which is exactly the order this issue itself proposed.

Cost for value: attesting the deterministic rule stage would prove something already provable more
cheaply. evaluateGateCheck is a pure function; the golden-corpus replay (#8832) and the
deterministic replay harness (#8838, hardened this same PR by closing #9135's unrecorded-holdout-draw
gap) already let anyone re-derive the identical verdict from the recorded findings + policy, with no
hardware and no KBS availability risk. Attestation's marginal value over replay is proving the CODE
VERSION that ran was the published one — a real claim, but it is a supply-chain/build-provenance
claim (reproducible builds, signed deploys), not one that needs the review pipeline wrapped in a
confidential VM on every PR.

3. Is a partial form (attest only the deterministic stage) sufficient?

Yes — and it is the only form worth reconsidering if this is ever picked up. evaluateGateCheck is
already pure and side-effect-free, so it is the cheapest possible thing to attest: short-lived, and
needing no live network access once unsealed (unlike the AI review call, which inherently talks to
an external provider). Keeping the model call outside the boundary is already the right design — its
non-determinism is the input #8838 records, and #9135 (this same PR) closes the one gap in that
record (the close-audit holdout draw) with an honest statement of what is still outside it
(precision-breaker flag state, the live CI aggregate, the global pause/freeze switches). Attesting an
outbound call to a third-party model provider would not meaningfully strengthen anything anyway — the
provider's own infrastructure stays unattested from our side regardless.

Recommendation

Not worth pursuing now.

• The trust gap attested live execution would close (the T3-adjacent "prove the review pipeline
  itself is honest") is not closed by attestation alone, per Research: is TEE hardware attestation necessary, or does a reproducible-backtest checksum approach cover the real threat? #8136's own conclusion — computation,
  not provenance.
• The cheaper anchoring + complete-records work in flight this same round (orb(trust): the 'public' decision-ledger verify endpoint requires an API token — 401 in prod, so the chain is verifiable by nobody #9120/orb(trust): the decision ledger has no external anchor and verify returns no hash — wholesale re-chaining and tail truncation both pass clean #9122/orb(trust): the decision record is never published — contributors get a 12-char prefix of a document no endpoint returns, and 51 preimages are already destroyed #9123/orb(trust): the decision record's commitments don't commit — modelId is always null, promptDigest ignores ten prompt suffixes, configDigest misses the calibrated close floor #9124/orb(trust): six of seven actuation sites write no decision record — cap, nag and approved-merge closes are invisible to the ledger and excluded from the certified guarantee #9134)
  closes the practical gap a network verifier cares about today.
• The live path's fail-open/fail-closed dilemma under a KBS outage is a strictly worse availability
  tradeoff than anything the offline backtest path ever faced, for a marginal proof replay already
  gives for free.
• If ever revisited, only the partial form (deterministic rule stage only, model call outside the
  boundary) is worth considering — full-path attestation including the AI call is not.

Revisit trigger: the same one #8540 already names for T3 generally — the first hosted tenant, or
an equivalent trust demand from the network, specifically asking for live-decision provenance beyond
an anchored, complete, replayable ledger. Until then this stays unscheduled, and the order this issue
proposed (complete records → external anchor → only then attested execution) stands with its third
step not started.

apps/loopover-ui/content/docs/verify-this-review.mdx now states this permanently — live-path
execution is out of scope for attestation by decision, not a temporary gap awaiting hardware.

[JSONbored]

Closing as answered — scope decision above: not worth pursuing now (attestation proves computation, not the record's own provenance; the anchoring + complete-records work in this same round closes the practical gap more cheaply and without the live path's fail-open/fail-closed availability tradeoff). Revisit trigger recorded above, matching #8540's T3 trigger. verify-this-review.mdx updated to state the boundary permanently.