Fix live materialization state-cache hits

CHANGELOG.md

@@ -7,31 +7,49 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Fixed
+
+- Live materialization now derives the current writer frontier, checks
+  `WarpStateCachePort` for exact and compatible predecessor snapshots before
+  replay, and publishes live replay results with their real WARP coordinate.
+- Exact state-cache hits now return cached materialization results without
+  republishing the same full snapshot back into the cache.
+- Live `wantDiff` materialization now bypasses state-cache hits so callers keep
+  receiving diff data from replay-backed materialization paths.
+- Receipt-producing materialization, including zero-patch empty results, now
+  bypasses state-cache reads and writes so callers keep receiving
+  replay-derived receipt arrays.
+- Live checkpoint fallback replay is now constrained to the requested frontier
+  coordinate before publishing a cache snapshot for that coordinate.
+- Live checkpoint reuse now verifies the checkpoint frontier is a predecessor
+  of the captured live coordinate before seeding state-cache publication.
+
+### Changed
+
+- Corrected CAS-first materialization documentation to describe the actual
+  WARP-owned state-cache lifecycle and its full-materialization memory limits.
+
 ## [18.2.0] - 2026-06-28
 
 ### Release notes
 
-`v18.2.0` introduces the streaming CAS-First Memoization Engine for graph
-materialization. By interrogating `git-cas` before initiating projection replay
-and utilizing a constant-memory pass-through `teeStream` during materialization,
-`git-warp` guarantees `O(1)` runtime memory footprints and zero-latency
-snapshot hydration.
+`v18.2.0` introduces the first state-cache memoization components for graph
+materialization. The release adds the initial cache port and `git-cas`-backed
+adapter groundwork for avoiding redundant projection replay, while keeping WARP
+frontier and snapshot semantics inside `git-warp`.
 
-It also enforces strict `@git-stunts/git-cas` boundary encapsulation across the
-repository, utilizing Buzhash Content-Defined Chunking (CDC) for automated
-rolling hash deduplication of unchanged sub-trees.
+It also starts enforcing `@git-stunts/git-cas` boundary encapsulation across the
+repository so WARP-owned state-cache payloads route through the storage library
+instead of hand-rolled CAS plumbing.
 
 ### Added
 
-- `CasFirstMemoizationEngine` now enforces the 3-step CAS memoization rule:
-  interrogate `git-cas` for existing snapshots, materialize via lazy streaming
-  upon a cache miss, and simultaneously pipe materialized buffers to `git-cas`
-  via `storeStream`.
+- Added initial CAS-first memoization and WARP state-cache components for
+  runtime integration work.
 - Added `has()` check capability to `BlobStoragePort` and `CasBlobAdapter` to
   support zero-latency CAS interrogations before buffering events.
 - Added `docs/topics/cas-first-memoized-materialization.md` to document the
-  constant-memory streaming materialization pipeline and Buzhash CDC rolling
-  hash deduplication mechanics.
+  state-cache materialization lifecycle and its current memory boundaries.
 
 ## [18.1.2] - 2026-06-25
 

docs/topics/README.md

@@ -33,9 +33,9 @@ inventories. Operator workflows live outside the topic shelf in
   checkpoints, replay, and provenance.
 - [Content and CAS](content-and-cas.md): handle content attachments,
   content-addressed storage, and encrypted CAS payloads.
-- [CAS-First memoized materialization](cas-first-memoized-materialization.md):
-  guarantee constant-memory streaming, Buzhash CDC deduplication, and
-  zero-latency snapshot hydration.
+- [WARP state-cache materialization](cas-first-memoized-materialization.md):
+  skip redundant live materialization replay through coordinate-addressed
+  state-cache snapshots backed by `git-cas`.
 - [Continuum boundary](continuum-boundary.md): understand what git-warp owns
   locally and what Continuum owns as boundary vocabulary.
 

docs/topics/cas-first-memoized-materialization.md

@@ -1,71 +1,92 @@
-# CAS-First Memoized Materialization
+# WARP State-Cache Materialization
 
-Use this page when you need to understand `git-warp`'s constant-memory streaming
-materialization pipeline, `@git-stunts/git-cas` boundary encapsulation, and
-rolling hash deduplication mechanics.
+Use this page when you need to understand how `git-warp` skips redundant
+materialization replay by memoizing WARP-owned state snapshots in
+`@git-stunts/git-cas`.
 
-## The Materialization Lifecycle
+`git-cas` is only the byte storage substrate. It does not know about WARP
+frontiers, optics, checkpoints, graph state, or materialization rules. `git-warp`
+owns those semantics through `WarpStateCachePort`; the Git-backed adapter stores
+snapshot payloads in `git-cas`.
 
-In `git-warp`, materialization is the bounded projection of raw CRDT graph
-events into structural checkpoints, working set views, or specialized hologram
-slices.
+## The Live Materialization Lifecycle
 
-To guarantee constant-memory `O(1)` runtime footprints and eliminate redundant
-CPU/memory computation across stigmergic peers, `git-warp` enforces a
-**CAS-First memoization pipeline**.
+When a Git-backed runtime has a state cache, live materialization follows this
+coordinate-first lifecycle:
 
 ```text
-+++++++> [git-cas] ---------> (materialization) ------> * (object)
-            ^                            |
-            |                            |
-            +----------------------------+
+[current frontier]
+        |
+        v
+[state-cache exact hit?] ---- yes ---> [return cached state]
+        |
+        no
+        v
+[compatible predecessor?] --- yes ---> [replay suffix, publish snapshot]
+        |
+        no
+        v
+[checkpoint/frontier replay] --------> [publish snapshot]
 ```
 
-## CAS-First Memoization Rules
+### 1. Derive a WARP coordinate
 
-Every materialization request must execute the following strict lifecycle:
+Before replay, the live path reads the current writer frontier and builds a
+WARP state coordinate:
 
-### 2.1. Is object already in git-cas?
+```text
+{ frontier: Map<writerId, tipSha>, ceiling: null }
+```
+
+This coordinate belongs to `git-warp`; it is not a `git-cas` concept.
+
+### 2. Check the WARP state cache
+
+The runtime asks `WarpStateCachePort` for an exact snapshot at that coordinate.
+On a hit, it returns the cached state without replaying writer patch streams and
+without republishing the same snapshot.
 
-Before executing any projection logic or buffering events into V8 heap memory,
-`git-warp` derives a deterministic materialization coordinate key:
-`key = sha256(baseFrontierSha + opticLensSha + queryParams)`.
+If no exact snapshot exists, the runtime asks for the best compatible
+predecessor. A predecessor hit lets materialization replay only the suffix after
+that cached coordinate, then publish a fresh snapshot for the current frontier.
 
-The runtime immediately interrogates `git-cas` (`await cas.has(key)`). If the
-object exists in storage, `git-warp` bypasses the entire projection calculation
-and streams the pre-calculated object directly to the caller.
+### 3. Fall back to replay and publish
 
-### 2.2. No? Materialize via streaming
+When there is no usable cached snapshot, the runtime falls back to the existing
+checkpoint/frontier replay path. Successful live and coordinate materializations
+publish an evictable state-cache snapshot with the actual coordinate so the next
+equivalent read can hit the cache.
 
-If the CAS interrogation returns a miss, `git-warp` initializes a lazy, chunked
-streaming materialization pipeline. Events are pulled from the underlying CRDT
-log in bounded batches, processed through the projection kernel, and immediately
-piped out to avoid accumulating unbounded memory buffers.
+## Memory Boundaries
 
-### 2.3. Write materialized git-object to git-cas always
+State-cache hits avoid redundant CRDT replay and can remove repeated startup
+costs for graph-sized materializations. They do not make legacy full
+materialization an `O(1)` memory API: a caller that asks for a full
+`SnapshotWarpState` still receives a full in-memory state object.
 
-As the object is materialized, the resulting buffer is simultaneously piped
-directly into `git-cas` (`cas.writeStream(key)`). This permanently memoizes the
-structural reality for all future causal code paths, background daemons, and
-remote peers.
+The bounded-memory read path is optic/worldline/query work over a sharded or
+streamed basis. The state cache is the replay-skipping compatibility bridge for
+legacy materialization and checkpoint flows.
 
-## Strict @git-stunts/git-cas Encapsulation
+## `git-cas` Encapsulation
 
-All CAS operations must route through the formal `@git-stunts/git-cas` library
-API. Direct invocation of raw git storage commands (`git hash-object`,
-`git cat-file`, `git mktree`) is strictly banned within `git-warp`.
+All state-cache payload storage routes through the formal `@git-stunts/git-cas`
+library API. Raw Git plumbing remains an adapter concern for WARP refs and Git
+object access; WARP state-cache payloads should not hand-roll a parallel CAS.
 
-### Buzhash Content-Defined Chunking (CDC)
+Routing state snapshots through `git-cas` allows content-addressed storage and
+chunk-level reuse where the underlying CAS representation can identify unchanged
+byte ranges. The WARP cache index remains responsible for determining whether a
+snapshot is semantically usable for a materialization coordinate.
 
-Routing through `@git-stunts/git-cas` unlocks advanced rolling hash capabilities:
+## Current Limitations
 
-- **Dynamic Chunking**: `@git-stunts/git-cas` employs a Buzhash rolling hash
-  algorithm to dynamically split streaming data into variable-length chunks
-  based on actual data content rather than fixed byte boundaries.
-- **Structural Deduplication**: If 99% of a materialized graph snapshot remains
-  unchanged between two consecutive frontiers, Buzhash CDC produces the exact
-  same block OIDs for the unchanged sub-trees. `@git-stunts/git-cas` instantly
-  deduplicates these blocks in memory before anything touches disk storage.
+- Exact state-cache hits bypass replay, but full materialization still hydrates
+  a full `WarpState`.
+- The Git-backed state-cache adapter stores full-state snapshots today. A future
+  sharded basis format should make optic reads avoid full-state hydration.
+- Cache coordinates must stay schema/version aware. A snapshot is reusable only
+  when WARP semantics say the coordinate is compatible.
 
 ## See also
 

src/domain/capabilities/PatchCollector.ts

@@ -27,10 +27,56 @@ function patchWithinCeiling(entry: PatchWithSha, ceiling: number | null): boolea
   return ceiling === null || entry.patch.lamport <= ceiling;
 }
 
+function patchAfterBaseCeiling(entry: PatchWithSha, ceiling: number | null): boolean {
+  return ceiling === null || entry.patch.lamport > ceiling;
+}
+
 function validTipSha(tipSha: string | undefined): tipSha is string {
   return typeof tipSha === 'string' && tipSha.length > 0;
 }
 
+function patchInCoordinateSuffix(
+  entry: PatchWithSha,
+  ceiling: number | null,
+  baseCeiling: number | null,
+): boolean {
+  return patchWithinCeiling(entry, ceiling) && patchAfterBaseCeiling(entry, baseCeiling);
+}
+
+function coordinateStopAtSha(
+  writerId: string,
+  baseCoordinate: { frontier: Map<string, string>; ceiling: number | null },
+): string | null {
+  if (baseCoordinate.ceiling !== null) {
+    return null;
+  }
+  const baseTipSha = baseCoordinate.frontier.get(writerId);
+  return validTipSha(baseTipSha) ? baseTipSha : null;
+}
+
+type PatchChainLoader = {
+  loadPatchChain(toSha: string, fromSha?: string | null): Promise<PatchWithSha[]>;
+};
+
+type CoordinateSuffixInput = {
+  writerId: string;
+  tipSha: string;
+  ceiling: number | null;
+  baseCoordinate: { frontier: Map<string, string>; ceiling: number | null };
+};
+
+async function* streamWriterCoordinateSuffix(
+  loader: PatchChainLoader,
+  input: CoordinateSuffixInput,
+): AsyncIterable<PatchWithSha> {
+  const stopAtSha = coordinateStopAtSha(input.writerId, input.baseCoordinate);
+  for (const entry of await loader.loadPatchChain(input.tipSha, stopAtSha)) {
+    if (patchInCoordinateSuffix(entry, input.ceiling, input.baseCoordinate.ceiling)) {
+      yield entry;
+    }
+  }
+}
+
 /**
  * Collects patches for materialization.
  *
@@ -84,9 +130,18 @@ export default abstract class PatchCollector {
   async *streamForFrontierSinceCoordinate(
     frontier: Map<string, string>,
     ceiling: number | null,
-    _baseCoordinate: { frontier: Map<string, string>; ceiling: number | null },
+    baseCoordinate: { frontier: Map<string, string>; ceiling: number | null },
   ): AsyncIterable<PatchWithSha> {
-    yield* this.streamForFrontier(frontier, ceiling);
+    for (const writerId of frontier.keys()) {
+      const tipSha = frontier.get(writerId);
+      if (!validTipSha(tipSha)) { continue; }
+      yield* streamWriterCoordinateSuffix(this, {
+        writerId,
+        tipSha,
+        ceiling,
+        baseCoordinate,
+      });
+    }
   }
 
   /** Load the latest checkpoint, or null if none. */
@@ -105,6 +160,9 @@ export default abstract class PatchCollector {
   /** Load a patch chain between two SHAs. */
   abstract loadPatchChain(_toSha: string, _fromSha?: string | null): Promise<PatchWithSha[]>;
 
+  /** Return whether one patch commit is an ancestor of another, when available. */
+  isAncestor?(_ancestorSha: string, _descendantSha: string): Promise<boolean>;
+
   /** Get the current writer frontier. */
   abstract getFrontier(): Promise<Map<string, string>>;
 }

src/domain/services/controllers/MaterializeController.ts

@@ -33,6 +33,10 @@ import MaterializePatchStreamReducer, {
   type MaterializePatchStreamReduction,
 } from './MaterializePatchStreamReducer.ts';
 import { summarizeMaterializePatches } from './MaterializePatchSummary.ts';
+import {
+  shouldPublishMaterializeSnapshot,
+  type MaterializeSnapshotPublicationOptions,
+} from './MaterializeSnapshotPublication.ts';
 import type LoggerPort from '../../../ports/LoggerPort.ts';
 import type CodecPort from '../../../ports/CodecPort.ts';
 import type CryptoPort from '../../../ports/CryptoPort.ts';
@@ -220,24 +224,28 @@ export default class MaterializeController {
   private async _emptyResult(
     ceiling?: number | null,
     frontier?: Map<string, string> | null,
+    options?: MaterializeSnapshotPublicationOptions,
   ): Promise<MaterializeResult> {
-    return await this._wrapState(createEmptyState(), ceiling ?? null, frontier ?? null);
+    return await this._wrapState(createEmptyState(), ceiling ?? null, frontier ?? null, options);
   }
 
   private async _wrapState(
     state: WarpState,
     ceiling: number | null,
     frontier: Map<string, string> | null,
+    options?: MaterializeSnapshotPublicationOptions,
   ): Promise<MaterializeResult> {
     const stateHash = await computeHash(this._deps, state);
     const adjacency = buildAdjacency(state);
-    await this._publishSnapshot({
-      state,
-      stateHash,
-      degraded: false,
-      ceiling,
-      frontier,
-    });
+    if (shouldPublishMaterializeSnapshot(options)) {
+      await this._publishSnapshot({
+        state,
+        stateHash,
+        degraded: false,
+        ceiling,
+        frontier,
+      });
+    }
     return {
       state,
       stateHash,
@@ -254,13 +262,15 @@ export default class MaterializeController {
   private async _buildResult(params: MaterializeResultBuildInput): Promise<MaterializeResult> {
     const stateHash = await computeHash(this._deps, params.reduced.state);
     const adjacency = params.reduced.adjacency ?? buildAdjacency(params.reduced.state);
-    await this._publishSnapshot({
-      state: params.reduced.state,
-      stateHash,
-      degraded: params.degraded,
-      ceiling: params.ceiling,
-      frontier: params.frontier,
-    });
+    if (params.reduced.receipts === undefined) {
+      await this._publishSnapshot({
+        state: params.reduced.state,
+        stateHash,
+        degraded: params.degraded,
+        ceiling: params.ceiling,
+        frontier: params.frontier,
+      });
+    }
     return {
       state: params.reduced.state,
       stateHash,
@@ -323,8 +333,10 @@ export default class MaterializeController {
   private _createStrategyRuntime(): MaterializeStrategyRuntime {
     return {
       deps: this._deps,
-      emptyResult: async (ceiling, frontier) => await this._emptyResult(ceiling, frontier),
-      wrapState: async (state, ceiling, frontier) => await this._wrapState(state, ceiling, frontier),
+      emptyResult: async (ceiling, frontier, options) =>
+        await this._emptyResult(ceiling, frontier, options),
+      wrapState: async (state, ceiling, frontier, options) =>
+        await this._wrapState(state, ceiling, frontier, options),
       reducePatches: async (patches, base, opts) => await this._reducePatches(patches, base, opts),
       reducePatchStream: async (stream, base, opts, provenanceBase) =>
         await this._reducePatchStream(stream, base, opts, provenanceBase),