feat(storage): add an authoritative durable small-write delta journal

[DecisionNerd]

Problem

Small graph mutations still pay immutable-snapshot write amplification. The existing adjacency delta segments are derived accelerators written after topology commits and may be ignored/rebuilt; they are not an authoritative mutation journal and cannot recover acknowledged graph changes.

The first #752 implementation exposed two correctness gaps now isolated in native children: .base_state.json became duplicate state authority instead of canonical Parquet, and the standalone publisher was not used by normal mutation paths while risking loss of non-graph participants. This canonical issue remains open until the complete-generation, typed-replay, and public-routing outcomes are proven.

Objective

Ship a checksummed, bounded, typed small-write delta path inside complete immutable generations, with CURRENT and the verified generation manifest as the only authority, lossless replay over canonical Parquet, and real Rust-facade mutation use.

Frozen ownership decision

• Every published generation is logically complete and owns its canonical Parquet base plus all authoritative typed delta runs beneath its own graph participant tree.
• No generation may depend on files in an ancestor generation; no cross-generation manifest DAG is introduced.
• .base_state.json, directory enumeration, adjacency deltas, caches, or mutation receipts cannot become graph-state authority.
• ADR 0019 must be amended before implementation diverges, and the shared reachability/GC oracle continues to operate over complete generations.

Requirements

• Amend the project-format/graph-capability ADR before introducing or changing authoritative delta bytes.
• Represent canonical Parquet plus immutable ordered typed delta runs as one manifest-verified complete graph generation.
• Frame records with version, transaction/operation identity, canonical UUID/surrogate identity, deterministic timestamps, labels/endpoints/relation type, property routing/types, ordering, length, and checksum.
• Make acknowledged journal entries durable under the frozen flush/CURRENT contract and shared filesystem admission.
• Preserve idempotent replay and stable conflict outcomes.
• Support the finite mutation set routed by feat(api): route eligible mutations through prepared delta publication #778 or explicitly select the existing full-Parquet path before mutation.
• Keep CURRENT and the generation manifest as the sole commit authority; never recover by scanning for the newest log.
• Bound run size/count, open validation, replay memory/work, affected rows, and corruption diagnostics with explicit configuration and typed failures.
• Make torn/corrupt/missing/reordered/duplicate/unsupported runs fail closed before partial query results.
• Keep adjacency delta accelerators derived and clearly separate from this authoritative journal.
• Preserve all non-graph capabilities/participants byte-for-byte when an eligible graph delta publishes.

Child ledger

Issue	Deliverable	Direct prerequisites
#777	Typed GFDR format and canonical-Parquet replay/materialization	#749, #776
#778	Prepared delta publication through eligible Rust-facade mutations	#777, #749, #776

Both children are native sub-issues and direct blockers of this canonical close gate.

Acceptance Criteria

• A small eligible public mutation publishes through one complete generation without rewriting unchanged base Parquet and is visible immediately and after reopen.
• Real canonical Parquet plus ordered typed runs reconstructs exact graph state without .base_state.json or another authority.
• Exact retry does not duplicate a mutation; conflicting identity reuse remains typed.
• Torn, truncated, reordered, duplicated, missing, checksum-invalid, unsupported, or resource-exceeding runs fail before partial visibility.
• Crash cases before and after acknowledgement match test(storage): model torn writes lost flushes and crash recovery deterministically #749 on a filesystem admitted by feat(storage): enforce filesystem durability preflight before project mutation #776.
• Every unrelated capability/participant remains byte-identical.
• Existing base-only v1 projects and the selected old-GFDR migration policy have deterministic regression evidence.
• feat(storage): replay typed graph deltas over canonical Parquet #777 and feat(api): route eligible mutations through prepared delta publication #778 are closed and their native relationships match this ledger.

BDD Completion Scenarios

• Given a real Parquet project with graph, ontology, knowledge, and provenance participants, when an eligible one-edge/property delta commits, then immediate and reopened Rust queries see it exactly once and unrelated participants remain intact.
• Given a checkpoint pins a pre-delta generation, when later complete generations publish typed runs, then the checkpoint remains unchanged and a fresh open sees the new state.
• Given a torn, missing, unsupported, or over-budget run, when GraphForge opens, then it fails closed before returning a partial graph.
• Given an unsupported or oversized mutation, when publication begins, then the existing full-Parquet path is selected before staging.

Implementation Notes

Canonical implementation is split between #777 and #778. Likely shared surfaces include graph_delta_journal.rs, schemas/catalog batched readers, graph/files participants, hydration/rematerialization, publish_graph_mutation_with_context, composite_publish.rs, GraphTransaction, mutation classification, and derived-index invalidation.

Observability

Rows/bytes/runs, format version, generation class, replay resource high-water marks, checksum phase, acknowledgement outcome, mutation classification, and safe transaction identity class only.

Security And Privacy

No graph values in logs. Validate lengths, counts, UUIDs, routing, types, and paths before allocation; keep every path machine-owned and contained.

Testing

Format goldens/property tests, real Parquet create/update/delete/property fixtures, public-facade immediate/reopen/checkpoint tests, participant preservation, fallback classification, idempotency/conflict cases, #749 fault histories, #776 admission, resource ladders, and thin binding/CLI parity. Provide deterministic fixture APIs consumed by #782 without treating benchmarks as correctness proof.

Documentation

Update ADR 0019, project format, concurrency/recovery, API behavior, mutation eligibility/fallback, migration policy, and testing/benchmark guidance.

Non-Goals

ARIES page logging, an unmanifested side WAL, cross-generation references, replacing Parquet as the compact base, routing every mutation family, or binding-side replay.

Related Issues

Canonical tracker #747. Direct prerequisites #749 and #776. Native children #777 and #778. Compaction #753. CodSpeed evidence #782.

[DecisionNerd]

Reopened as part of the M6 closure recovery after an exact-tree audit of 6019f8a9b837b1b34615baad220fe6933c540b7d.

The current journal is not integrated with GraphForge's real graph state. load_base_state reads graph/deltas/.base_state.json and returns an empty graph when that test sidecar is absent, while production graph readers continue to read canonical Parquet without replaying .gfdr runs. Repository-wide call-site inspection found no normal GraphForge mutation/query path that publishes and observes the authoritative delta journal.

Recovery exit:

• derive the base state from canonical Parquet rather than a test-only JSON authority;
• wire acknowledged deltas into the Rust-owned mutation and read paths so normal GraphForge reopen/query observes them;
• prove a real v1 Parquet project plus create/update/delete deltas through facade-level reopen/query tests;
• retain idempotency, fail-closed corruption, and CURRENT-only authority.

#752 continues to block #753, #756, and canonical tracker #747.