fix(storage): bound manifest allocation with authenticated bucket leaves

[DecisionNerd]

Maintainer scope: before v1.0.0

Backward compatibility is out of scope for this issue and its repair sub-issues before the v1.0.0 release. Do not add or retain legacy readers, old API aliases, compatibility shims, mixed-version support, migration/backfill machinery, or old-version regression tests solely to preserve behavior or data from earlier GraphForge versions. Earlier backward-compatibility requirements in this issue are superseded by this maintainer instruction.

This does not authorize unrelated breaking changes or weaken the issue's current-version acceptance criteria. Exact current-format semantics, supported current-version API/binding and export/import interoperability, authentication, corruption/unsupported-format refusal, crash recovery, retry/idempotency, active snapshots, cancellation and resource budgets remain required where applicable. Never silently reinterpret unsupported old data. Document intentional format/API breaks and the supported current format; a migration implementation is not required. Current-version correctness tests and explicitly behavior-preserving refactors remain in scope. Compatibility guarantees for v1.0.0 and later are a separate release-policy decision.

Problem and evidence

#1196's heterogeneous fixture has 544 authenticated payload entries represented by 750 Patricia nodes occupying 3,072,000 allocated bytes. A test-only bounded bucket representation preserves authenticated lookup of every entry using 238 objects and estimates 974,848 bytes at 4-KiB allocation granularity. Existing one-entry leaves amplify small-file allocation.

Scope and acceptance

• Implement versioned bounded bucket leaves with at most eight entries and an explicit serialized/decoded-byte bound. Preserve each path, length, kind and SHA and authenticated copy-on-write lookup.
• Incremental insertion, replacement, deletion, 8→9 split and merge/collapse preserve current-format entries without scanning or rewriting the entire manifest. Backward compatibility, mixed legacy trees and migration are not required (maintainer instruction: pre-v1).
• Reject malformed prefixes, duplicate or misplaced entries, oversized buckets, digest corruption and unsupported collisions. Prove successful and absent lookups and strict work bounds.
• Retain active generations/snapshots and existing publication recovery/durability/lease guarantees.
• On test(storage): establish permanent topology and identity compaction budgets #1196's fixed heterogeneous fixture, actual current manifest-node allocation is at most half the 3,072,000-byte legacy baseline on 4-KiB native storage; report total project bytes separately. Keep deterministic update/read resource budgets and exact facade reopen/query/export/full-verify/clean-import tests.
• Focused PR, required exact-head CI/CI Gate, appropriate local checks and documented current format and source-bound measurements.

Given current-format manifest nodes, when a bounded bucket child is published and queried, then both active snapshots authenticate exact file metadata. Given insert/delete at a bucket boundary or interrupted publication, lookup and recovery preserve the previous or complete new authority, never a partial tree.

Native child of #1194, blocked by assessment #1196; blocks #1194. Excludes payload codecs, generic metadata frameworks, unrelated M3 and S24/S26 certification. Debt: allocation amplification.

Verified production result

Implemented in #1239. The fixed 352-entry current fixture uses 209 authenticated bucket/branch objects and 856,064 allocated manifest bytes, down from 483 objects and 1,978,368 bytes (56.7%). This also meets the historical #1196 acceptance budget of 1,536,000 bytes; the historical 544-entry denominator is reported separately. Lookup/update/split/collapse, malformed-input refusal and current-format size limits have direct tests. Public construction/mutation, immediate query, reopen, export/full verification, clean import and subsequent mutation pass across all 45 facade fixtures.

Source-bound resource evidence is committed in docs/development/evidence/bounded-manifest-1204.json. It distinguishes attributed permanent bytes, the point-in-time whole-project census, process RSS and sampled overlapping file allocations. Whole-fixture CPU observations are not an isolated manifest speedup claim; syscall writes rose 58,054 bytes while syscall reads fell 2,574,887 bytes. No S20/S22 integrated evidence or final capacity claim is made here.

Local formatting, Clippy, pre-push-fast and gate-registry checks passed. Final release facade (45/45) and scale accounting (30/30) suites passed. Full local pre-push failed solely at the existing hardcoded /tmp filesystem-admission unit test; this failure remains documented rather than skipped or weakened.

Squash merge c74a529ac9c60997c1343b9f36a074bef39173d8 completed on 2026-09-10. Required exact-head Test Suite/CI Gate run 34515324325 passed at 86f80f51e55beba6838f72db7223ae24f7a07077, including authoritative Bazel, both bindings, native Windows/macOS durability and concurrency. The PR was CLEAN, had no unresolved review threads and closed exactly #1204. Issue closure was verified at 19:02:36 UTC.

[DecisionNerd]

Production bounded buckets are implemented locally on fix/1204-bounded-manifest at 36360cb8 (not merged; issue remains open).

The fixed #1196 heterogeneous public fixture passes with the unchanged semantic fingerprint. Actual production manifest allocation is 856,064 bytes / 209 objects / 352 entries, meeting the 1,536,000-byte acceptance budget. The direct current baseline is 1,978,368 bytes / 483 objects / 352 entries, a 1,122,304-byte (56.7%) reduction. Historical #1196 remains separately recorded: 3,072,000 bytes / 750 objects / 544 entries; 192 edge payload references were already removed by earlier merged fragmentation changes and are not attributed to this repair.

Ten manifest parser/corruption tests and all 39 object-store tests passed. The added 128/256/512-entry resource ladder passed with three replacement reads/installs and three deletion reads at each size. Eight-to-nine splitting, exact-root collapse and retained-root authentication pass. Independent read-only review found no verified blocker.

The frozen candidate executable has completed its first CPU/RSS and public construction/reopen/query/export/full-verification/clean-import run. Comparable syscall tracing and filesystem peak sampling are underway; subsequent-mutation/full recovery suite, final local gates, required exact-head CI and merge remain outstanding. No whole-process hard-memory or query-speed improvement is claimed from the logical manifest budgets.

[DecisionNerd]

Evidence relocated from the docs tree (#1625). The 1 file(s) below were committed under docs/development/ as measurement evidence for this issue. They are removed from the tree by #1628 (the #1625 pull request); the repository keeps method and content digests, results live here. Every file remains available at its permalink on commit 29a7b34ebe44, and its SHA-256 is recorded next to it.

Files (1 total, 377 lines)

• docs/development/evidence/bounded-manifest-1204.json sha256:6dd88c16df1e02ae3d8166dc3f3ba06ec4d1ad0012e378143046bbac4885996b