Skip to content

perf(storage): pack current construction endpoint and identity records #1274

Description

@DecisionNerd

Verified cause and bounded opportunity

Current construction records retain padding that is written, sorted, merged, authenticated and read throughout shaping. Endpoint records use 48 bytes for 33 bytes of fields; resolved endpoints use 32 for 25; base/shaped identities use 32 for 26. Source and independent review verified the full UUIDs, kind, role, surrogate and required retained-identity marker. Packing saves 30 logical bytes per edge in each complete endpoint stream, 14 per edge in each resolved stream and 6 per identity in each base/shaped stream. These are record-volume estimates, not additive simultaneous-peak claims.

The #1268 frozen final 1x/2x/4x fixture supplies a complete baseline after consumed-root retirement: peaks 18,513,920 / 35,979,264 / 70,909,952 bytes; shape reads 54,544,436 / 124,245,316 / 282,652,710; merge writes 37,339,136 / 89,112,576 / 208,519,168. Three runs observed CPU of 4.73–4.89 seconds and maximum RSS of 49,976 KiB. Preserve these executable-bound observations and failed/superseded evidence.

Scope

Pack endpoint records from 48 to 33 bytes, resolved endpoints from 32 to 25, and base/shaped identities from 32 to 26. Preserve 128-bit UUIDs, role/kind, retained marker and 64-bit surrogate. Update every producer/reader, receipt-size admission, accounting and current deterministic budget. The permanent UUID-index encoder is a verified downstream consumer; its permanent 25-byte output stays unchanged and excludes the construction marker.

Use exact private construction format 10. Reject earlier checkpoints before any recovery mutation. Per canonical #1194 maintainer scope, do not add or retain old-version construction readers, success fixtures or migration machinery solely for compatibility. Preserve substantive current-format lifecycle, corruption, recovery, cancellation, retry and snapshot assertions. Historical measurements retain their original layouts. No generic writer abstraction, permanent encoding-policy change or new certification workflow.

Acceptance criteria

  • Implement all three packed records across production publication and UUID-index consumers; preserve exact current semantics and corruption/unsupported-format refusal.
  • Prove deterministic 1x/2x/4x allocated-peak and logical read/write reductions against the frozen perf(storage): retire consumed shaping roots before resolved merge peak #1268 baseline. Preselect each peak budget as baseline minus 14 bytes per edge (one resolved-stream padding contribution); logical shape reads and merge writes must fall at every scale. Measure actual coexistence rather than summing streams.
  • Freeze executables before three resource observations. Report CPU, kernel/process read/write I/O, process RSS, retained/peak allocations and limitations. Preselected CPU diagnostic ceiling: 5.21 seconds (baseline maximum 4.89 plus twice its 0.16 span). RSS ceiling: 54,072 KiB (baseline 49,976 plus a 4,096 KiB explicit allowance, not a hard bound).
  • Exercise actual public construction/mutation, immediate query, reopen, export, full verification, clean import and subsequent mutation with exact UUIDs, properties and surrogate allocation. Preserve interrupted shaping/encoding/publication, cancellation, retries and active snapshots. Test truncated/malformed packed records and unsupported old-format refusal before mutation.
  • Pass changed-surface checks, independent review and exact-head CI Gate; squash merge and verify closure. Update the existing integrated report. Final merged host evidence reuses the existing admitted S18/S19/S20/S22 ladder with unchanged host/reserve; no S24/S26 execution.

Completion scenario

Given current-format construction with multiple merge levels and retained parent identities, when packed runs pass through shaping, publication and portable interchange, exact graph identities and properties survive while temporary allocation and merge I/O fall. Malformed or unsupported authority is refused without changing CURRENT or recovery controls.

Focused native child and blocker of #1194. Canonical #1194/#900/#745 retain final named-host capacity/S26 acceptance.

Verified implementation completion

PR #1275, head 62b7cde88d17349f827ae2a0637342840824d63e, implements the bounded format change and records the paired comparison in docs/development/evidence/packed-construction-records-1274.json. Peaks fall 17.26–17.81%, logical shape reads 15.80–16.52% and merge writes 20.36–20.59% beyond #1268. Three release-profile observations pass all preselected budgets; observed maximum RSS rises 492 KiB and remains below 54,072 KiB. No causal CPU/latency claim is made.

Local frozen storage suite: 1,105 passed, one known #1192 /tmp filesystem-class failure, two existing ignored. Public construction lifecycle 5/5, four publishing-path contracts 4/4, packed golden/refusal tests 4/4, workspace Clippy, formatting, fast checks and gate-registry validation passed. Initial failures and excluded debug/failed-freeze evidence are retained. Exact-head CI 34881765747 passed at the exact PR head; all required checks were green with CLEAN merge state and zero unresolved review threads.

This issue owns packing implementation and its paired/direct lifecycle evidence. Final same-host integrated S18/S19/S20/S22 evidence necessarily follows the implementation merge and remains tracked by canonical #1194; it does not require S24/S26 or a new certification gate before this implementation can merge.

Squash merge verified: 710c6c64f4718c0664d08bdd3aafe21de4a29eba (PR #1275, 2026-09-14). This issue is CLOSED; merged tree equals the tested PR tree. Measured release executable and complete successful CI log are retained.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    coreCore source code changesenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions