Skip to content

fix(storage): remove redundant authenticated construction passes #971

Description

@DecisionNerd

Problem

The append-only construction model is in place, but the immediate construction-to-publication path still repeats full authenticated file passes across seal, shape, encoding, CAS installation, and hydration. Attempt 5 showed this constant-factor amplification dominates ingest time even while RSS stays bounded.

Objective

Remove redundant whole-payload passes from the ordinary authenticated construction path while preserving fail-closed integrity, crash recovery, atomic publication, and bounded application I/O.

Parent scope

This is the deterministic code-repair slice of #901. It does not close #901: #951 and S20/S22 still own actual allocated/peak-disk, RSS, elapsed-time, and provider evidence.

Acceptance criteria

  • Immediate seal and shape authenticate staged fixed-run artifacts while consuming them; no prior blanket payload pass.
  • Completed shaped writers persist authenticated digest/length/identity capabilities so the normal encoding path does not rehash their payloads; recovery reauthenticates when no completed capability exists.
  • Newly installed CAS objects use one source copy-and-hash pass into a fresh CAS-owned inode, with fsync, readonly/identity checks, namespace durability, and fail-closed reuse/race handling.
  • Hydration performs one installed-object verification and does not redundantly verify the same immutable source inode.
  • Deterministic failure boundaries prove retry never publishes corrupt or lost payload state.
  • Payload-dominated 1x/2x/4x tests reconcile every application-read phase, canonical output, and staged/retained logical bytes with constant-factor bounds.
  • Documentation labels application-observed I/O honestly and routes actual physical/allocated/peak disk evidence to test(scale): attribute physical bytes per node and edge before SCALE26 #951.

BDD scenarios

Constant-factor authenticated publication

Given equivalent payload-dominated construction runs at 1x, 2x, and 4x scale
When the graph is sealed, shaped, encoded, installed, and hydrated
Then application bytes for each payload-owned phase grow approximately 2x per doubling
And the sum exactly reconciles with the phase evidence.

Crash-safe CAS installation

Given failure after any durable CAS installation boundary
When construction is retried
Then no manifest references corrupt or missing content
And retry produces a cryptographically verified object without losing the accepted payload.

Writer capability recovery

Given a shaped writer completes and durably records its capability
When encoding consumes that output
Then it does not reopen the whole payload merely to recreate a receipt
But missing or incomplete capability state forces fail-closed reauthentication.

Non-goals

Provider execution, actual device-level physical I/O measurement, S20/S22 certification, changing integrity policy, or weakening filesystem admission.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions