Skip to content

A historical failed-append leaves /verify permanently red with no way to declare it #9933

Description

@JSONbored

The public ledger verifier on the production ORB reports ok:false and cannot be made honest-green, because one of its two findings has no declaration mechanism.

Measured on the production ORB (orb-v3.7.0-beta.1, 2069 ledger rows)

1. 83 content mismatches — declarable today. All in seq 5..257, all in schemaVersion 2 and 3; every one of the 1,569 v5 rows verifies clean. I recomputed the digests offline with the repo's own canonicalJson/contentDigest and confirmed the count and range exactly, then brute-forced every subset of null-valued keys per row (the obvious "stored as null, hashed as undefined" hypothesis) — no subset reproduces the committed digest for any of the 83. Their preimages are genuinely unrecoverable, which is precisely what LOOPOVER_LEDGER_CONTENT_WAIVER (#9850) exists for.

2. 231 unchained records — NOT declarable. decision_records rows with no decision_ledger entry: the failed-append signature. Spanning 2026-07-04T15:18:19Z .. 2026-07-24T18:25:38Z, and nothing since 07-24, so this is historical residue from a durability bug that has stopped, not an ongoing fault. The chain itself is intact — the tip chains correctly (2069.prev_hash == 2068.row_hash) and the scan reports no sequence_gap, predecessor_mismatch, or row_hash_mismatch anywhere.

The gap

The content waiver deliberately never applies to chain findings — correctly, since a waived row that is mis-chained must still fail. But that means an unchained_record has no declaration path at all. The result is a public integrity endpoint stuck at ok:false forever, for a known, bounded, already-fixed historical cause. That is the worst possible state for an integrity signal: permanently red is indistinguishable from newly-broken, so a REAL failed append tomorrow is invisible.

Proposed fix

A declared, bounded, disclosed exclusion for unchained records, mirroring LOOPOVER_LEDGER_CONTENT_WAIVER exactly in spirit:

  • Bounded by created_at, not seq — an orphan has no ledger row and therefore no sequence number.
  • Both bounds and a reason mandatory; malformed fails CLOSED and is surfaced by preflight, same as ledger: /verify reports ok:false — content_mismatch at seq 5 on a 3-day-old chain #9850.
  • Waived orphans counted and published separately (waivedUnchainedRecords + unchainedWaiver), never folded into the real count, and always disclosed even on a clean chain.
  • Genuinely structural findings (sequence_gap / predecessor_mismatch / row_hash_mismatch / short_tail) stay unwaivable. A truncated tail is exactly the attack this check defends, so only the INTERIOR unchained_record kind is declarable.

That restores the property that matters: red means something new is wrong.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions