Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 108 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,17 @@ metadata exposure, see [docs/THREAT_MODEL.md](./docs/THREAT_MODEL.md).

`git-cas` uses Git as the storage substrate, not as a user-facing abstraction.

At a high level, the system does four things:
At a high level, the system does five things:

1. turns input bytes into chunk blobs stored in Git
2. records how to rebuild those bytes in a manifest
3. emits a Git tree that keeps the manifest and chunk blobs reachable
4. optionally indexes trees by slug through a GC-safe vault ref
3. exposes assets, pages, and bundles through opaque application handles
4. retains or publishes handle graphs through explicit lifecycle policies
5. inspects repository reachability and managed-storage health without mutation

Low-level blob, manifest, and tree operations remain available. Application
code can use the higher-level capabilities without constructing Git trees,
persisting naked object IDs, or implementing its own cache ref protocol.

The same core supports:

Expand Down Expand Up @@ -81,6 +86,27 @@ credential resolution share `bin/credentials.js` so raw key files,
passphrase-derived vault keys, and encrypted restore input requirements stay
consistent across human and machine command surfaces.

Application-facing lifecycle capabilities sit above the same CAS and Git
ports:

```mermaid
flowchart TD
APP[Application]
HANDLE[Assets / Pages / Bundles]
POLICY[Retention / Publication / Cache / Expiring Set]
CAS[CAS Pipeline]
GIT[Git Objects and Refs]
DOCTOR[Repository Diagnostics]

APP --> HANDLE
HANDLE --> CAS
HANDLE --> POLICY
CAS --> GIT
POLICY --> GIT
DOCTOR --> GIT
APP --> DOCTOR
```

## Store Pipeline

```
Expand Down Expand Up @@ -117,6 +143,10 @@ The public entrypoint is [index.js](./index.js).
observability adapters
- exposes convenience methods like `storeFile()` and `restoreFile()`
- exposes `rootSets.open()` for mutable current-generation Git reachability
- exposes opaque `assets`, `pages`, and `bundles` capability groups
- exposes `retention`, `publications`, `caches`, and `expiringSets` for explicit
reachability and lifecycle policy
- exposes read-only repository evidence through `diagnostics.doctor()`

The facade is orchestration glue. It is not the storage engine itself.

Expand Down Expand Up @@ -145,6 +175,28 @@ The facade is orchestration glue. It is not the storage engine itself.
`RootSetPersistence`, `RootSetMetadataCodec`, and `RootSetTreeCodec` own the
ref, metadata, and real Git reachability boundaries.

- **`AssetService`, `PageService`, and `BundleService`** — stage application
payloads behind immutable opaque handles. Pages enforce bounded blob size;
bundles build deterministic bounded-fanout trees and support targeted member
traversal without hydrating the complete structure.

- **`RetentionService` and `PublicationService`** — validate complete handle
graphs, then either retain them in a RootSet with generation-scoped evidence
or publish them atomically under an allowlisted application ref.

- **`CacheSetRegistry` and `CacheSet`** — own cache index generations,
compare-and-swap replacement, TTL, entry and logical-byte limits,
approximate-LRU sweeps, coalesced touches, doctor, and authoritative repair
under `refs/cas/caches/*`.

- **`ExpiringSetRegistry` and `ExpiringSet`** — own digest-only replay markers
under `refs/cas/expiring/*`. Admission is atomic, membership reads are
side-effect free, and only expired entries can be released.

- **`RepositoryDoctor`** — streams object, ref, reflog-reachability, and safe
prune-dry-run evidence to classify repository objects and summarize managed
CacheSet, RootSet, ExpiringSet, and Vault usage.

- **`KeyResolver`** — resolves key sources: passphrase-derived keys via KDF,
envelope recipient DEK wrapping and unwrapping. `CasService` delegates all key
material resolution through `this.keyResolver`.
Expand Down Expand Up @@ -200,6 +252,14 @@ The facade is orchestration glue. It is not the storage engine itself.
- **`Oid`** — immutable Git object identifier value with 40- or 64-hex
validation.
- **`RootSetRef`** — immutable validated ref below `refs/cas/rootsets/`.
- **`AssetHandle`, `PageHandle`, and `BundleHandle`** — canonical repository-
independent locators for validated application payload graphs.
- **`RetentionWitness`** — immutable evidence naming the generation and real
Git tree edge that retained or published a handle graph.
- **`CacheKey`, `CachePolicy`, `CacheHit`, and `CacheSetRef`** — validated cache
identity, policy, lookup evidence, and namespace boundaries.
- **`ExpiringMarker`, `ExpiringSetKey`, and `ExpiringSetRef`** — validated
replay-marker evidence, domain-separated key identity, and namespace refs.
- **`Slug`** — immutable vault slug representation. It enforces vault slug
limits and exposes `.toTreePath()` for the percent-encoded Git tree-entry name
used by plain vault trees.
Expand Down Expand Up @@ -249,6 +309,8 @@ with methods that throw "not implemented" by default.
- **`CompressionPort`** — compress/decompress for both byte arrays and streams.
- **`ObservabilityPort`** — metrics, logs, and spans without binding the domain
to any runtime event API.
- **`RepositoryInspectionPort`** — non-mutating streams for objects, refs,
reachable IDs, safe prunable candidates, and reachable physical bytes.

### Infrastructure (`src/infrastructure/`)

Expand All @@ -267,6 +329,9 @@ Git:
`@git-stunts/plumbing` to shell out to the `git` CLI.
- **`GitRefAdapter`** — `GitRefPort` implementation using
`@git-stunts/plumbing`.
- **`GitRepositoryInspectionAdapter`** — `RepositoryInspectionPort`
implementation using streamed Git inventories and plumbing's non-mutating
prunable-object inspection.

Compression:
- **`NodeCompressionAdapter`** — `CompressionPort` backed by `node:zlib`.
Expand Down Expand Up @@ -362,6 +427,46 @@ For Merkle assets the tree contains:
Chunk blobs are deduplicated at the tree-entry level by digest. The manifest
still remains authoritative for repeated-chunk order and multiplicity.

### Application Handles And Structured Materializations

An opaque handle is a canonical content locator, not a durability claim. Asset
handles identify existing CAS trees. Page handles identify bounded immutable
blobs. Bundle handles identify deterministic descriptor trees whose leaves are
assets, pages, or nested bundles.

Bundle traversal reads only the descriptor path and selected payload required
for a named member. Validation re-enforces persisted path, descriptor-byte,
fanout, and nesting limits before a graph is retained or published.

### Retention And Publication

RootSets, CacheSets, ExpiringSets, application publications, and the Vault all
make objects Git-reachable, but they encode different lifecycle contracts:

| Surface | Ref namespace | Lifecycle |
| --- | --- | --- |
| RootSet | `refs/cas/rootsets/*` | Caller-managed current generation |
| CacheSet | `refs/cas/caches/*` | TTL/capacity/approximate-LRU managed cache |
| ExpiringSet | `refs/cas/expiring/*` | Expiry-only replay protection |
| Publication | Caller-allowlisted ref | Application-controlled causal history |
| Vault | `refs/cas/vault` | History-preserving named assets |

High-level retention, CacheSet, ExpiringSet, and application publication
operations return generation-scoped retention evidence. Replacing a parentless
current-generation head releases old edges when no other ref or reflog reaches
them; Git's normal grace and pruning policy still controls physical deletion.

### Repository Diagnostics

`cas.diagnostics.doctor()` reports total, anchored, orphaned, volatile, and
unreachable object evidence plus managed-storage summaries. Ref and reflog
reachability defines anchored objects; volatile objects come only from safe
prune dry-run output for the selected expiry cutoff.

The report names limits Git cannot prove exactly, including per-owner
deduplicated physical bytes and packed-object age. The inspection surface has
no object-write, ref-update, GC, or destructive prune operation.

### Vault

The vault is a GC-safe slug index rooted at `refs/cas/vault`.
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [6.2.0] — 2026-07-13

### Added

- **Application storage ownership contract** — cycle API-0047 defines opaque
Expand Down Expand Up @@ -62,6 +64,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- **6.2.0 release verification** — `pnpm run release:verify` passed all 13
steps with 6,124 observed tests across Node, Bun, Deno, and all three
real-Git integration runtimes. npm and JSR metadata now agree on `6.2.0`;
build stamping, npm package inspection, and the JSR publish dry-run passed.
- **Safe prune inspection dependency** — repository diagnostics require
`@git-stunts/plumbing` 3.1.0, whose public API fixes the exact non-mutating
`git prune --dry-run --verbose --no-progress --expire=<cutoff>` contract.
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Unlike traditional LFS which moves files to external servers, `git-cas` treats t
Existing v5 users should read [UPGRADING.md](./UPGRADING.md) and run
`npm run upgrade` in dry-run mode before restoring old encrypted vault entries.
For the release overview, see the
[v6.1.0 Release Notes](./docs/releases/v6.1.0.md).
[v6.2.0 Release Notes](./docs/releases/v6.2.0.md).

### 1. CLI Usage

Expand Down Expand Up @@ -183,6 +183,9 @@ All three runtimes are tested in CI on every push. The hexagonal architecture is
- **[v6.0.0 Release Notes](./docs/releases/v6.0.0.md)**: Major-release summary, upgrade call-to-action, and publication notes.
- **[v6.1.0 Release Notes](./docs/releases/v6.1.0.md)**: Root-set retention,
doctor/repair, and GC-policy guidance.
- **[v6.2.0 Release Notes](./docs/releases/v6.2.0.md)**: Opaque application
storage, managed cache and replay lifecycles, retention evidence, and
repository diagnostics.
- **[Upgrading](./UPGRADING.md)**: Migration guide for v5 → v6.
- **[Changelog](./CHANGELOG.md)**: Version history and migration notes.

Expand Down
12 changes: 6 additions & 6 deletions STATUS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# STATUS

**Last tagged release:** `v6.1.0` (`2026-07-11`)
**Current release state:** `v6.1.0` is published to npm and GitHub Releases; JSR publication remains deferred behind the upstream toolchain gate.
**Latest verification:** `npm run release:verify -- --skip-jsr` passed 12/12 steps with 5,521 observed tests on `2026-07-11`; release run `29170112655` also passed lint, unit tests, Node/Bun/Deno integration, npm trusted publication, and GitHub Release creation.
**Current release state:** `v6.2.0` release candidate; npm and GitHub Release publication remain pending the reviewed tag workflow.
**Latest verification:** `pnpm run release:verify` passed 13/13 steps with 6,124 observed tests on `2026-07-13`, including Node/Bun/Deno unit and integration suites, npm package inspection, and a JSR publish dry-run.
**Playback truth:** `main`
**Runtimes:** Node.js 22.x, Bun, Deno
**Current planning method:** [WORKFLOW.md](./WORKFLOW.md)
Expand All @@ -18,10 +18,10 @@
- The machine-facing `git cas agent` surface exists and now supports
OS-keychain passphrase sources for vault-derived key flows, but parity and
portability are still partial.
- **v6.1.0 artifact posture** — npm publication and the GitHub Release are
complete through tag `v6.1.0`. JSR publication is deferred because the current
`jsr`/Deno toolchain panics before package validation; it can return in a
later 6.x maintenance change once its dry-run is healthy.
- **v6.2.0 artifact posture** — the versioned release candidate and npm package
surface are verified locally. npm trusted publication and GitHub Release
creation remain pending the `v6.2.0` tag workflow. JSR dry-run validation is
healthy, but JSR publication is not part of this release workflow.
- **v6.0.0 encryption scheme simplification** — `whole-v1`/`whole-v2` collapsed
to `whole`, `framed-v1`/`framed-v2` collapsed to `framed`, `convergent-v1`
collapsed to `convergent`. AAD is now always on. Legacy scheme strings in
Expand Down
43 changes: 43 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,49 @@

v6.0.0 is a major release that simplifies the encryption model, hardens security defaults, and cleans up the architecture. This guide covers every breaking change and what you need to do.

## v6.1.0 To v6.2.0

v6.2.0 is API-additive and does not require stored-data migration. It adds
high-level application storage, managed cache and replay lifecycles, immutable
retention evidence, and repository-wide diagnostics.

Applications should stop treating a naked Git object ID stored in JSON as a
durability guarantee. Stage content through `assets`, `pages`, or `bundles`,
then retain or publish the returned opaque handle through the policy surface
that matches its lifecycle:

```javascript
const staged = await cas.assets.put({
source,
slug: 'warp/materialization',
filename: 'materialization.bin',
});

const cache = await cas.caches.open({
namespace: 'git-warp/materializations',
policy: { maxEntries: 10_000, maxBytes: 2 * 1024 * 1024 * 1024 },
});
const stored = await cache.put(cacheKey, staged.handle, {
retention: 'evictable',
expiresAt: new Date(Date.now() + 14 * 24 * 60 * 60 * 1000),
});
```

Existing root sets and vault entries remain valid. Before running destructive
Git cleanup, applications migrating an existing object-ID index must first
enumerate every still-live object, validate that it still exists, and adopt it
into a RootSet or rebuild it through the appropriate high-level API. Neither
repair nor diagnostics can recover an object that Git already pruned.

Use `cas.diagnostics.doctor()` before and after migration to inspect anchored,
orphaned, and grace-expired object counts plus managed CacheSet, RootSet,
ExpiringSet, and Vault usage. The doctor is read-only; it never runs Git GC or
prune. Repository diagnostics require `@git-stunts/plumbing` 3.1.0 or newer.

See [v6.2.0 Release Notes](./docs/releases/v6.2.0.md) and
[Application Storage](./docs/API.md#application-storage) for the complete
capability and policy map.

## v6.0.1 To v6.1.0

v6.1.0 is API-additive and does not require stored-data migration. It adds root
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ release_home: 'v6.2.0'
issue: 'https://github.com/git-stunts/git-cas/issues/58'
goalpost_issue: 'https://github.com/git-stunts/git-cas/issues/50'
tracker_source: 'github'
status: 'active'
status: 'landed'
base_commit: '295f21fd5b4d73d353ee36647804b96686329157'
owners:
- '@git-stunts'
Expand Down Expand Up @@ -800,10 +800,19 @@ recorded in [witness/asset-handles.md](./witness/asset-handles.md).
Structured bundle and page implementation evidence is recorded in
[witness/bundle-pages.md](./witness/bundle-pages.md).

Managed CacheSet lifecycle evidence is recorded in
[witness/cache-set.md](./witness/cache-set.md).

Expiry-safe replay-set evidence is recorded in
[witness/expiring-set.md](./witness/expiring-set.md).

Repository-wide object, reachability, managed-storage, and non-mutation
evidence is recorded in
[witness/repository-diagnostics.md](./witness/repository-diagnostics.md).

Version, package, documentation, and local release-gate evidence is recorded in
[witness/release-candidate.md](./witness/release-candidate.md).

## Risks

- The facade could become a new monolith. Mitigation: capability groups delegate
Expand Down Expand Up @@ -850,4 +859,34 @@ evidence is recorded in

## Retrospective

Pending implementation and release playback.
What changed from the design:

- The shipped names stayed capability-oriented: `assets`, `pages`, `bundles`,
`retention`, `publications`, `caches`, `expiringSets`, and `diagnostics`.
- Cache metadata uses immutable parentless generations and a bounded
approximate-LRU policy rather than a mutable side database.
- Replay protection received a distinct ExpiringSet surface because capacity
or recency eviction would violate the minimum retention window.
- Repository diagnostics report exact evidence only where Git can prove it;
shared-byte attribution and packed-object age remain explicit unknowns.

What the tests proved:

- Application payloads can be staged and streamed through opaque handles,
retained with generation-scoped witnesses, or atomically published under an
allowlisted application ref.
- Bundles and pages support bounded targeted traversal without materializing a
complete structured value.
- Cache replacement preserves the newly published generation under concurrent
updates, and replay markers cannot be released before expiry.
- Repository diagnostics include refs and reflogs, use only dry-run prune
inspection, fail closed on inconsistent inventories, and do not mutate Git.

What remains open:

- Applications such as git-warp must migrate their own domain indexes and
causal publication rules onto these capabilities.
- A future major release may reorganize legacy root-level plumbing without
removing the deliberate low-level escape hatch.
- Very large repository diagnostics may later use commit-graph or bitmap
acceleration without changing the evidence contract.
Loading
Loading