diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ab7c05c..a5c25ac5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -53,9 +53,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 bounded inspection, doctor evidence, and immutable retention witnesses. The capability intentionally exposes no live remove, repair, capacity, or recency eviction path. +- **Repository-wide diagnostics** — `cas.diagnostics.doctor()` streams all Git + objects, refs, and reflog reachability; classifies anchored, orphaned, and + dry-run-prunable loose objects; reports repository physical bytes where Git + can prove them; and summarizes bounded CacheSet, RootSet, ExpiringSet, and + Vault usage. Per-owner physical bytes and packed-object age remain explicit + unknowns, and concurrent inventory changes fail closed. ### Changed +- **Safe prune inspection dependency** — repository diagnostics require + `@git-stunts/plumbing` 3.1.0, whose public API fixes the exact non-mutating + `git prune --dry-run --verbose --no-progress --expire=` contract. + - **Git object size port** — `GitPersistencePort.readObjectSize()` and the Git adapter use object metadata to enforce page bounds without materializing content. diff --git a/README.md b/README.md index 6dacc246..c60d9b23 100644 --- a/README.md +++ b/README.md @@ -104,6 +104,7 @@ The README is the front door. Detailed mechanics live in the guide set: | GC retention for caches and derived state | [Root Sets](./docs/API.md#root-sets) | | Managed TTL and capacity caches | [Cache Sets](./docs/API.md#cache-sets) | | Durable replay markers with expiry-only release | [Expiring Sets](./docs/API.md#expiring-sets) | +| Repository reachability and git-cas usage evidence | [Repository Diagnostics](./docs/API.md#repository-diagnostics) | | v5 to v6 migration | [Upgrading](./UPGRADING.md) | Core capabilities: @@ -134,8 +135,10 @@ Core capabilities: compare-and-swap refs, and immutable lifecycle evidence. - **Envelope recipients**: multi-recipient key wrapping and recipient rotation avoid re-encrypting data blobs. -- **Operational diagnostics**: `git-cas doctor` validates vault health and - reports deduplication efficiency. +- **Operational diagnostics**: `cas.diagnostics.doctor()` streams repository + object/ref evidence, classifies anchored, orphaned, and volatile objects, + and summarizes CacheSet, RootSet, ExpiringSet, and Vault usage without + mutating Git. The `git-cas doctor` CLI continues to validate vault health. ## Safety Snapshot diff --git a/docs/API.md b/docs/API.md index 792143fd..cedfdb82 100644 --- a/docs/API.md +++ b/docs/API.md @@ -17,13 +17,16 @@ should treat restored data, chunker output, codec output, and keys as 1. [ContentAddressableStore](#contentaddressablestore) 2. [Application Storage](#application-storage) 3. [Root Sets](#root-sets) -4. [Vault](#vault) -5. [CasService](#casservice) -6. [Events](#events) -7. [Value Objects](#value-objects) -8. [Ports](#ports) -9. [Codecs](#codecs) -10. [Error Codes](#error-codes) +4. [Cache Sets](#cache-sets) +5. [Expiring Sets](#expiring-sets) +6. [Repository Diagnostics](#repository-diagnostics) +7. [Vault](#vault) +8. [CasService](#casservice) +9. [Events](#events) +10. [Value Objects](#value-objects) +11. [Ports](#ports) +12. [Codecs](#codecs) +13. [Error Codes](#error-codes) ## ContentAddressableStore @@ -1369,8 +1372,9 @@ Returns a non-mutating report with: - stable issue codes for missing or mismatched targets Current members are `anchored`. `orphaned` and `volatile` describe objects -outside the current root set and are reported as zero by a single-set doctor; -repository-wide classification remains a Git maintenance concern. +outside the current root set and are reported as zero by a single-set doctor. +Use [`cas.diagnostics.doctor()`](#repository-diagnostics) for repository-wide +classification across refs and reflogs. ### repair @@ -1681,6 +1685,69 @@ must validate request timestamps and choose `expiresAt` from the same trusted clock. Marker state survives process restart because the Git ref, not process memory, is authoritative. +## Repository Diagnostics + +`cas.diagnostics.doctor()` returns immutable, machine-readable evidence about +the complete Git object store and the managed git-cas refs without running +garbage collection or destructive prune. + +```javascript +const report = await cas.diagnostics.doctor({ + gracePeriodMs: 14 * 24 * 60 * 60 * 1000, + maxCollectionsPerKind: 100, +}); + +console.log(report.repository.objects); +console.log(report.usage.caches); +console.log(report.limitations); +``` + +Pass either `gracePeriodMs` or an exact canonical `expiresBefore` UTC timestamp, +not both. The default grace period is 14 days. `maxCollectionsPerKind` bounds +detailed CacheSet, RootSet, and ExpiringSet rows from 1 through 1000; its default +is 100. Every managed collection is still inspected sequentially and included +in `totals`. Coverage reports `observed`, `inspected`, `detailed`, and +`complete`, so detail truncation is visible instead of silently dropping +managed refs or undercounting repository usage. + +The reachability classes have precise operational meanings: + +- `anchored` is the object set reached by all refs and all reflog entries. +- `volatile` is the loose unreachable set reported by + `git prune --dry-run --verbose --no-progress --expire=`. +- `orphaned` is the remaining unreachable set not targeted by that exact dry + run. + +The volatile inventory always goes through +`GitPlumbing.inspectPrunableObjects()` from `@git-stunts/plumbing`; git-cas does +not construct or expose a mutating prune path. If independent writers change +the object store while the streamed inventories run, arithmetic invariants +fail closed: `healthy` becomes false, derived counts become `null`, and the +report includes `REPOSITORY_CHANGED_DURING_INSPECTION`. + +Repository-wide physical bytes are reported where Git provides compatible +evidence: total object bytes, anchored bytes, and their combined unreachable +difference. Physical bytes for an individual cache, root set, vault, orphaned +class, or volatile class are `null`; shared objects and pack deltas cannot be +assigned honestly to one owner. A dry prune also cannot expose the age of +packed unreachable objects. Git may include configured alternate object stores +in its inventory, while object disk sizes exclude pack indexes, bitmaps, and +other repository metadata. These facts appear in `limitations` rather than +being estimated. + +Cache summaries report entry count, deterministic logical bytes, age, expiry, +capacity policy, and pinned/evictable counts. RootSet policy counts and Vault +entry counts are reported independently from reachability. A privacy-mode +vault remains healthy but reports `entryCount: null` because repository doctor +does not request or retain vault key material. + +Object and ref inventories are consumed as streams, and managed collections are +inspected one at a time. Runtime memory is bounded by stream windows, +`maxCollectionsPerKind`, and existing per-collection/blob safety limits; +runtime is linear in repository object and managed-collection count. Doctor +never updates a ref, writes an object, runs `git gc`, or runs a destructive +`git prune`. + ## Vault The vault provides GC-safe storage by maintaining a single Git ref (`refs/cas/vault`) pointing to a commit chain. The commit's tree indexes all stored assets by slug. This prevents `git gc` from garbage-collecting stored data. @@ -2571,6 +2638,31 @@ class CustomGitAdapter { } ``` +### RepositoryInspectionPort + +The domain-facing, non-mutating port behind repository doctor. Its Git adapter +streams all-object metadata, reachable OIDs, dry-run-prunable loose objects, +and refs, and obtains the aggregate reachable object disk usage. Application +code normally uses `cas.diagnostics.doctor()` instead of this low-level port. + +```javascript +for await (const object of port.iterateObjects()) { + console.log(object.oid, object.type, object.logicalBytes, object.physicalBytes); +} + +for await (const oid of port.iterateReachableObjectIds()) { + // refs and reflogs both contribute +} + +for await (const object of port.iteratePrunableObjects({ expiresBefore })) { + // safe dry-run evidence only +} +``` + +`GitRepositoryInspectionAdapter` requires `@git-stunts/plumbing` 3.1.0 or +newer. Structured Git output is validated strictly; malformed records or a +non-zero stream status fail with `REPOSITORY_INSPECTION_INVALID`. + ### CodecPort Interface for encoding/decoding manifest data. @@ -2987,6 +3079,7 @@ new CasError({ message, code, meta, documentationUrl }); | `ROOT_SET_METADATA_INVALID` | `.rootset.json` is malformed, non-canonical, or belongs to another ref | `read()`, `list()`, `doctor()` | | `ROOT_SET_TREE_INVALID` | Metadata and the Git tree's actual reachability edges disagree | `read()`, `list()`, `doctor()` | | `ROOT_SET_REF_UPDATE_FAILED` | Root-set ref update failed for a non-conflict reason | Root-set mutations and repair | +| `REPOSITORY_INSPECTION_INVALID` | Repository doctor options, Git output, dependencies, or safe-integer totals are invalid | `cas.diagnostics.doctor()`, repository inspection adapter | | `VAULT_ENTRY_NOT_FOUND` | Slug does not exist in vault | `removeFromVault()`, `resolveVaultEntry()` | | `VAULT_ENTRY_EXISTS` | Slug already exists (use `force` to overwrite) | `addToVault()` | | `VAULT_CONFLICT` | Concurrent vault update detected (CAS failure after retries) | `addToVault()`, `removeFromVault()`, `initVault()`, `rotateVaultPassphrase()` | diff --git a/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md b/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md index f826ead2..741da68e 100644 --- a/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md +++ b/docs/design/0047-application-storage-cache-boundary/application-storage-cache-boundary.md @@ -800,6 +800,10 @@ recorded in [witness/asset-handles.md](./witness/asset-handles.md). Structured bundle and page implementation evidence is recorded in [witness/bundle-pages.md](./witness/bundle-pages.md). +Repository-wide object, reachability, managed-storage, and non-mutation +evidence is recorded in +[witness/repository-diagnostics.md](./witness/repository-diagnostics.md). + ## Risks - The facade could become a new monolith. Mitigation: capability groups delegate diff --git a/docs/design/0047-application-storage-cache-boundary/witness/repository-diagnostics.md b/docs/design/0047-application-storage-cache-boundary/witness/repository-diagnostics.md new file mode 100644 index 00000000..84361962 --- /dev/null +++ b/docs/design/0047-application-storage-cache-boundary/witness/repository-diagnostics.md @@ -0,0 +1,181 @@ +# API-0047 Repository Diagnostics Witness + +Implementation slices: + +- [#49](https://github.com/git-stunts/git-cas/issues/49) +- [#55](https://github.com/git-stunts/git-cas/issues/55) + +Implementation commit: `868cc7080513f8fed71f28dd93bb744ebdf0fc85` + +Review-hardening commit: `6ba961ab2aefb704c2e3d9c56a71d161d67b77be` + +## Claim + +`git-cas` owns non-mutating repository-wide CAS diagnostics. The public facade +streams total object, reachable-object, prunable-object, and ref evidence; +classifies anchored, orphaned, and volatile objects under an explicit grace +policy; and reports CacheSet, RootSet, ExpiringSet, and Vault usage without +claiming physical-byte attribution that Git cannot prove. + +## Source Evidence + +- The frozen `cas.diagnostics.doctor()` capability initializes its focused + service lazily, after the existing Git-backed collection registries and vault + are available. + [cite: `index.js#184-196@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + [cite: `index.js#306-318@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + [cite: `index.js#404-410@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- The repository port exposes only streamed inventories and one reachable-byte + query. It has no object-write, ref-update, garbage-collection, or destructive + prune method. + [cite: `src/ports/RepositoryInspectionPort.js#1-33@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- The Git adapter streams structured object metadata, all ref-and-reflog + reachable OIDs, refs, and reachable disk usage. Volatile inspection delegates + exclusively to `GitPlumbing.inspectPrunableObjects()`. + [cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#29-100@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- Stream parsing handles arbitrary chunk boundaries, validates OIDs, object + types, and safe-integer byte counts, and rejects nonzero Git completion. + [cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#103-161@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- RepositoryDoctor derives unreachable and orphaned counts by checked + arithmetic, keeps physical attribution unknown where classes overlap shared + storage, and fails closed when concurrent repository writes violate its + inventory invariants. + [cite: `src/domain/services/RepositoryDoctor.js#25-68@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + [cite: `src/domain/services/RepositoryDoctor.js#71-105@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- Managed refs are consumed one at a time. Every observed collection is + inspected and included in health and totals; `maxCollectionsPerKind` caps + only retained detail rows. + [cite: `src/domain/services/RepositoryDoctor.js#108-163@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + [cite: `src/domain/services/RepositoryDoctor.js#399-489@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +- Known Git evidence limits are first-class report data: shared physical bytes, + packed-object age, reflog cardinality, alternate stores, pack metadata, and + detail truncation are never silently estimated. + [cite: `src/domain/services/RepositoryDoctor.js#539-583@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +## Real Git Evidence + +The Docker-gated integration proof creates a bare repository containing a live +branch, annotated tag, reflog-only commit, CacheSet, RootSet, ExpiringSet, +Vault entry, recent unreachable blob, and old unreachable blob. It verifies +that the reflog object remains anchored, the recent blob remains orphaned, the +old blob is volatile under the exact cutoff, managed usage is visible, and the +complete object-ID and ref inventories are byte-for-byte unchanged after +doctor runs. + +[cite: `test/integration/repository-diagnostics.test.js#67-109@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/integration/repository-diagnostics.test.js#115-168@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +## Self-Review + +The implementation was reviewed against #49, #55, and API-0047 for command +safety, reachability semantics, reflog coverage, grace policy, bounded memory, +shared-byte attribution, concurrent writes, privacy, immutable output, runtime +portability, and additive semver posture. + +- Repository diagnostics expose no mutation primitive. +- Git output is parsed structurally instead of inferred from prose. +- Full object and ref inventories stream; managed collections are inspected + sequentially; only bounded detail rows remain resident. +- Reachability, retention policy, expiry, logical bytes, and physical bytes stay + separate dimensions. +- Unknown or unprovable values are `null` with explicit limitations. +- Graft reports only additive source symbols and no removed or changed export. +- Graft structural/reference coverage finds test references for + `RepositoryDoctor`, `GitRepositoryInspectionAdapter`, and + `RepositoryInspectionPort`. + +## Code Lawyer Review + +### CL-001: A detail cap could silently undercount repository usage + +All managed refs are inspected sequentially and update aggregate health and +totals before the detail-row cap is applied. Coverage distinguishes observed, +inspected, and detailed cardinality. +[cite: `src/domain/services/RepositoryDoctor.js#108-134@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `src/domain/services/RepositoryDoctor.js#428-480@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/domain/services/RepositoryDoctor.test.js#179-214@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-002: Diagnostics could accidentally expose destructive prune + +The adapter receives only the safe plumbing inspection stream. The port cannot +write objects, update refs, run GC, or request destructive prune. +[cite: `src/ports/RepositoryInspectionPort.js#1-33@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#57-71@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-003: Independent writers could make subtraction look authoritative + +Negative or unsafe derived counts make the report unhealthy, replace derived +counts with `null`, and emit `REPOSITORY_CHANGED_DURING_INSPECTION`. +[cite: `src/domain/services/RepositoryDoctor.js#32-42@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `src/domain/services/RepositoryDoctor.js#80-103@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/domain/services/RepositoryDoctor.test.js#240-265@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-004: Deduplicated physical bytes could be double-counted by owner + +Per-cache, per-root-set, per-vault, orphaned, and volatile physical bytes remain +unknown. The report states why, while repository total, anchored, and combined +unreachable bytes use compatible Git evidence. +[cite: `src/domain/services/RepositoryDoctor.js#90-104@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `src/domain/services/RepositoryDoctor.js#539-560@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-005: Chunked or failed Git output could be partially trusted + +The line decoder preserves partial chunks until delimiters arrive, flushes the +decoder, validates every structured field, and checks stream completion status. +Unit tests cover arbitrary boundaries, malformed output, and nonzero status. +[cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#103-161@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js#34-58@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js#114-144@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-006: A safe integer can still exceed the JavaScript Date range + +Grace policy validates both integer shape and the computed timestamp before +calling `toISOString()`, preserving the documented CAS error surface. +[cite: `src/domain/services/RepositoryDoctor.js#259-272@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/domain/services/RepositoryDoctor.test.js#267-287@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-007: Repository doctor could demand or leak a privacy-vault key + +Privacy metadata returns healthy but unknown entry cardinality and an explicit +limitation without invoking `readState()`. +[cite: `src/domain/services/RepositoryDoctor.js#165-205@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/domain/services/RepositoryDoctor.test.js#216-238@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +### CL-008: Eager diagnostics wiring could break existing facade adapters + +The facade installs a frozen forwarding capability immediately but constructs +the adapter and doctor only after the first diagnostics request and after the +ordinary service registries exist. +[cite: `index.js#184-196@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `index.js#404-410@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] +[cite: `test/unit/facade/ContentAddressableStore.diagnostics.test.js#21-68@868cc7080513f8fed71f28dd93bb744ebdf0fc85`] + +## Pull Request Review Follow-up + +CodeRabbit identified three valid hardening opportunities, all resolved before +merge: + +- Failed CacheSet, RootSet, and ExpiringSet inspections now return distinct + error records that exactly match their public usage contracts while + preserving structured error evidence. + [cite: `src/domain/services/RepositoryDoctor.js#136-163@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] + [cite: `src/domain/services/RepositoryDoctor.js#514-550@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] + [cite: `test/unit/domain/services/RepositoryDoctor.test.js#217-259@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] +- The buffered full reachable-byte inventory has a dedicated five-minute + default policy instead of inheriting the short stream-start timeout. An + explicitly injected caller policy remains authoritative. + [cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#6-28@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] + [cite: `src/infrastructure/adapters/GitRepositoryInspectionAdapter.js#91-97@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] +- Negative-path tests now prove that refs outside `refs/` and malformed + prunable-object records fail closed with `REPOSITORY_INSPECTION_INVALID`. + [cite: `test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js#114-140@6ba961ab2aefb704c2e3d9c56a71d161d67b77be`] + +## Verification + +- Node unit: 212 files, 1,866 passed, 2 skipped. +- Bun unit: 211 files passed, 1 skipped; 1,865 passed, 3 skipped. +- Deno unit: 1,856 passed under the runtime-filtered suite. +- Real-Git integration: 174 passed on each of Node, Bun, and Deno. +- ESLint and `git diff --check` passed. +- `pnpm run release:verify`: 13/13 steps, 6,109 tests observed, npm and + JSR publish dry-runs passed. diff --git a/index.d.ts b/index.d.ts index 106e6314..884c828e 100644 --- a/index.d.ts +++ b/index.d.ts @@ -528,6 +528,31 @@ export declare class GitRefAdapter extends GitRefPortBase { constructor(options: { plumbing: unknown; policy?: unknown }); } +export type RepositoryObjectType = 'blob' | 'tree' | 'commit' | 'tag'; + +export interface RepositoryObjectRecord { + readonly oid: string; + readonly type: RepositoryObjectType; + readonly logicalBytes: number; + readonly physicalBytes: number; +} + +/** Abstract non-mutating repository inspection port. */ +export declare class RepositoryInspectionPort { + iterateObjects(): AsyncIterable; + iterateReachableObjectIds(): AsyncIterable; + iteratePrunableObjects(options: { + expiresBefore: string; + }): AsyncIterable>; + iterateRefs(): AsyncIterable<{ readonly ref: string; readonly oid: string }>; + reachablePhysicalBytes(): Promise; +} + +/** Git-backed implementation of repository inspection. */ +export declare class GitRepositoryInspectionAdapter extends RepositoryInspectionPort { + constructor(options: { plumbing: unknown; policy?: unknown }); +} + /** Node.js crypto implementation of CryptoPort. */ export declare class NodeCryptoAdapter extends CryptoPortBase { constructor(); @@ -1037,6 +1062,192 @@ export declare class VaultService { getVaultMetadata(): Promise; } +export interface RepositoryDoctorOptions { + /** Canonical UTC cutoff passed to safe prune inspection. Mutually exclusive with gracePeriodMs. */ + expiresBefore?: string; + /** Relative grace period. Defaults to 14 days. Mutually exclusive with expiresBefore. */ + gracePeriodMs?: number; + /** Maximum detailed CacheSet, RootSet, and ExpiringSet reports per kind. @default 100 */ + maxCollectionsPerKind?: number; +} + +export interface RepositoryObjectMetric { + readonly objectCount: number | null; + readonly physicalBytes: number | null; +} + +export interface RepositoryObjectInventory { + readonly total: RepositoryObjectMetric & { readonly logicalBytes: number }; + readonly anchored: RepositoryObjectMetric; + readonly orphaned: RepositoryObjectMetric; + readonly volatile: RepositoryObjectMetric; + readonly unreachable: RepositoryObjectMetric; +} + +export interface RepositoryCollectionCoverage { + readonly observed: number; + readonly inspected: number; + readonly detailed: number; + readonly complete: boolean; +} + +export interface RepositoryDiagnosticLimitation { + readonly code: string; + readonly message: string; + readonly kind?: 'caches' | 'rootSets' | 'expiringSets'; + readonly observed?: number; + readonly inspected?: number; + readonly detailed?: number; +} + +export interface RepositoryCacheUsage { + readonly namespace: string; + readonly ref: string; + readonly generation: string; + readonly healthy: boolean; + readonly entryCount: number | null; + readonly logicalBytes: number | null; + readonly physicalBytes: null; + readonly retention: { + readonly pinnedEntries: number | null; + readonly evictableEntries: number | null; + } | null; + readonly reachability: 'anchored'; + readonly age: { + readonly createdAt: string | null; + readonly updatedAt: string | null; + readonly oldestAccessedAt: string | null; + } | null; + readonly expiry: { + readonly expiredEntries: number; + readonly nextExpiry: string | null; + } | null; + readonly policy: Readonly | null; + readonly issues: ReadonlyArray>; +} + +export interface RepositoryRootSetUsage { + readonly ref: string; + readonly generation: string; + readonly healthy: boolean; + readonly entryCount: number | null; + readonly physicalBytes: null; + readonly retention: { + readonly pinnedEntries: number | null; + readonly evictableEntries: number | null; + } | null; + readonly reachability: RootSetDoctorResult['reachabilityCounts'] | 'anchored' | null; + readonly issues: ReadonlyArray>; +} + +export interface RepositoryExpiringSetUsage { + readonly namespace: string; + readonly ref: string; + readonly generation: string; + readonly healthy: boolean; + readonly entryCount: number | null; + readonly physicalBytes: null; + readonly reachability: 'anchored'; + readonly age: { + readonly createdAt: string | null; + readonly updatedAt: string | null; + } | null; + readonly expiry: { + readonly liveEntries: number; + readonly expiredEntries: number; + readonly nextExpiry: string | null; + } | null; + readonly issues: ReadonlyArray>; +} + +export interface RepositoryVaultUsage { + readonly ref: string; + readonly present: boolean; + readonly healthy: boolean; + readonly generation: string | null; + readonly entryCount: number | null; + readonly physicalBytes: null; + readonly privacy: boolean | null; + readonly reachability: 'anchored' | null; + readonly issues: ReadonlyArray>; +} + +export interface RepositoryDoctorReport { + readonly version: 1; + readonly healthy: boolean; + readonly observedAt: string; + readonly completedAt: string; + readonly policy: { + readonly gracePeriodMs: number | null; + readonly expiresBefore: string; + readonly maxCollectionsPerKind: number; + }; + readonly repository: { + readonly objects: RepositoryObjectInventory; + readonly roots: { + readonly refCount: number; + readonly reflogsIncluded: true; + readonly reflogCount: null; + }; + readonly evidence: { + readonly anchoredInventory: 'refs-and-reflogs'; + readonly prunableInspection: 'dry-run'; + readonly mutatesRepository: false; + }; + }; + readonly usage: { + readonly caches: { + readonly healthy: boolean; + readonly coverage: RepositoryCollectionCoverage; + readonly totals: { + readonly entryCount: number | null; + readonly logicalBytes: number | null; + readonly pinnedEntries: number | null; + readonly evictableEntries: number | null; + readonly expiredEntries: number | null; + }; + readonly entries: ReadonlyArray; + }; + readonly rootSets: { + readonly healthy: boolean; + readonly coverage: RepositoryCollectionCoverage; + readonly totals: { + readonly entryCount: number | null; + readonly pinnedEntries: number | null; + readonly evictableEntries: number | null; + }; + readonly entries: ReadonlyArray; + }; + readonly expiringSets: { + readonly healthy: boolean; + readonly coverage: RepositoryCollectionCoverage; + readonly totals: { + readonly entryCount: number | null; + readonly liveEntries: number | null; + readonly expiredEntries: number | null; + }; + readonly entries: ReadonlyArray; + }; + readonly vault: RepositoryVaultUsage; + }; + readonly limitations: ReadonlyArray; +} + +/** Repository-wide, non-mutating diagnostics domain service. */ +export declare class RepositoryDoctor { + constructor(options: { + repository: RepositoryInspectionPort; + rootSets: { open(options: { ref: string }): RootSet | Promise }; + caches: { open(options: { namespace: string }): CacheSet | Promise }; + expiringSets: { + open(options: { namespace: string }): ExpiringSet | Promise; + }; + vault: Pick; + clock?: { now(): Date }; + }); + doctor(options?: RepositoryDoctorOptions): Promise; +} + /** Result of comparing two manifests by chunk digest. */ export interface ManifestDiffResult { added: Chunk[]; @@ -1148,6 +1359,10 @@ export interface ExpiringSetCapability { }): Promise; } +export interface DiagnosticsCapability { + doctor(options?: RepositoryDoctorOptions): Promise; +} + export interface RetentionResult { readonly changed: boolean; readonly witness: RetentionWitness; @@ -1197,6 +1412,7 @@ export default class ContentAddressableStore { readonly caches: CacheCapability; readonly expiringSets: ExpiringSetCapability; + readonly diagnostics: DiagnosticsCapability; readonly assets: AssetCapability; readonly pages: PageCapability; diff --git a/index.js b/index.js index e83c0f60..8cd471f8 100644 --- a/index.js +++ b/index.js @@ -16,12 +16,14 @@ import CacheSet from './src/domain/services/CacheSet.js'; import CacheSetRegistry from './src/domain/services/CacheSetRegistry.js'; import ExpiringSet from './src/domain/services/ExpiringSet.js'; import ExpiringSetRegistry from './src/domain/services/ExpiringSetRegistry.js'; +import RepositoryDoctor from './src/domain/services/RepositoryDoctor.js'; import PageService from './src/domain/services/PageService.js'; import PublicationService from './src/domain/services/PublicationService.js'; import RetentionService from './src/domain/services/RetentionService.js'; import rotateVaultPassphrase from './src/domain/services/rotateVaultPassphrase.js'; import GitPersistenceAdapter from './src/infrastructure/adapters/GitPersistenceAdapter.js'; import GitRefAdapter from './src/infrastructure/adapters/GitRefAdapter.js'; +import GitRepositoryInspectionAdapter from './src/infrastructure/adapters/GitRepositoryInspectionAdapter.js'; import createCryptoAdapter from './src/infrastructure/adapters/createCryptoAdapter.js'; import { createGitPlumbing } from './src/infrastructure/createGitPlumbing.js'; import { storeFile, restoreFile } from './src/infrastructure/adapters/FileIOHelper.js'; @@ -67,8 +69,10 @@ export { RootSetRegistry, CacheSet, ExpiringSet, + RepositoryDoctor, GitPersistenceAdapter, GitRefAdapter, + GitRepositoryInspectionAdapter, JsonCodec, CborCodec, SilentObserver, @@ -99,6 +103,7 @@ export { default as StatsCollector } from './src/infrastructure/adapters/StatsCo export { default as FixedChunker } from './src/infrastructure/chunkers/FixedChunker.js'; export { default as CdcChunker } from './src/infrastructure/chunkers/CdcChunker.js'; export { default as CompressionPort } from './src/ports/CompressionPort.js'; +export { default as RepositoryInspectionPort } from './src/ports/RepositoryInspectionPort.js'; export { default as NodeCompressionAdapter } from './src/infrastructure/adapters/NodeCompressionAdapter.js'; export { default as diffManifests } from './src/domain/services/ManifestDiff.js'; export { SCHEME_WHOLE, SCHEME_FRAMED, SCHEME_CONVERGENT } from './src/domain/encryption/schemes.js'; @@ -186,6 +191,9 @@ export default class ContentAddressableStore { this.expiringSets = Object.freeze({ open: async (options) => (await this.#getExpiringSetRegistry()).open(options), }); + this.diagnostics = Object.freeze({ + doctor: async (options) => (await this.#getRepositoryDoctor()).doctor(options), + }); this.assets = Object.freeze({ put: async (options) => (await this.#getAssetService()).put(options), adopt: async (options) => (await this.#getAssetService()).adopt(options), @@ -231,6 +239,8 @@ export default class ContentAddressableStore { #cacheSetRegistry = null; /** @type {ExpiringSetRegistry|null} */ #expiringSetRegistry = null; + /** @type {RepositoryDoctor|null} */ + #repositoryDoctor = null; #servicePromise = null; /** @@ -293,6 +303,21 @@ export default class ContentAddressableStore { return this.service; } + #initRepositoryDoctor() { + const cfg = this.#config; + this.#repositoryDoctor = new RepositoryDoctor({ + repository: new GitRepositoryInspectionAdapter({ + plumbing: cfg.plumbing, + policy: cfg.policy, + }), + rootSets: this.#rootSetRegistry, + caches: this.#cacheSetRegistry, + expiringSets: this.#expiringSetRegistry, + vault: this.#vault, + clock: cfg.clock, + }); + } + #initCollectionServices({ persistence, ref, crypto, cfg }) { this.#cacheSetRegistry = new CacheSetRegistry({ persistence, @@ -376,6 +401,15 @@ export default class ContentAddressableStore { return this.#expiringSetRegistry; } + /** @returns {Promise} */ + async #getRepositoryDoctor() { + await this.#getService(); + if (this.#repositoryDoctor === null) { + this.#initRepositoryDoctor(); + } + return this.#repositoryDoctor; + } + /** @returns {Promise} */ async #getAssetService() { await this.#getService(); diff --git a/package.json b/package.json index eac961a5..29d5e5c5 100644 --- a/package.json +++ b/package.json @@ -98,7 +98,7 @@ "@flyingrobots/bijou-tui": "^5.0.0", "@flyingrobots/bijou-tui-app": "^5.0.0", "@git-stunts/alfred": "^0.10.0", - "@git-stunts/plumbing": "^3.0.3", + "@git-stunts/plumbing": "^3.1.0", "@git-stunts/vault": "^1.0.1", "cbor-x": "^1.6.0", "commander": "14.0.3", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6f75394f..838a7092 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -24,8 +24,8 @@ importers: specifier: ^0.10.0 version: 0.10.0 '@git-stunts/plumbing': - specifier: ^3.0.3 - version: 3.0.3 + specifier: ^3.1.0 + version: 3.1.0 '@git-stunts/vault': specifier: ^1.0.1 version: 1.0.1 @@ -297,8 +297,8 @@ packages: resolution: {integrity: sha512-0DPhJdKhYTcsPuoOnYIIyvlwaIM7yIx4fQM4Q48abe/VDLTfZef1ubeT1pYio+ZTp1lKXtSG663973ewBbi/yw==} engines: {node: '>=20.0.0'} - '@git-stunts/plumbing@3.0.3': - resolution: {integrity: sha512-LmPaq9c51bYA1Ta0KJrmewFTlAwueQv4EdguiI4/TNK9LOeJkkchC2RaOX09WXubauvUeEk8Z32rG8zxdk491g==} + '@git-stunts/plumbing@3.1.0': + resolution: {integrity: sha512-SLRySIS8FyTOy3mV43GY0xp/1QPu+Q/5CAbDE1HmL+NfnMhuMe6stiycjCQW10WLzbI6ihP+WKqrCgoMfAdRTw==} engines: {bun: '>=1.3.5', deno: '>=2.0.0', node: '>=20.0.0'} '@git-stunts/vault@1.0.1': @@ -1161,7 +1161,7 @@ snapshots: '@git-stunts/alfred@0.10.0': {} - '@git-stunts/plumbing@3.0.3': + '@git-stunts/plumbing@3.1.0': dependencies: zod: 3.25.76 diff --git a/src/domain/errors/Codes.js b/src/domain/errors/Codes.js index fb7445a8..00714fa9 100644 --- a/src/domain/errors/Codes.js +++ b/src/domain/errors/Codes.js @@ -60,6 +60,7 @@ const ErrorCodes = Object.freeze({ PUBLICATION_REF_UPDATE_FAILED: 'PUBLICATION_REF_UPDATE_FAILED', RECIPIENT_ALREADY_EXISTS: 'RECIPIENT_ALREADY_EXISTS', RECIPIENT_NOT_FOUND: 'RECIPIENT_NOT_FOUND', + REPOSITORY_INSPECTION_INVALID: 'REPOSITORY_INSPECTION_INVALID', RESTORE_TOO_LARGE: 'RESTORE_TOO_LARGE', RETENTION_WITNESS_INVALID: 'RETENTION_WITNESS_INVALID', ROOT_SET_CONFLICT: 'ROOT_SET_CONFLICT', diff --git a/src/domain/services/RepositoryDoctor.js b/src/domain/services/RepositoryDoctor.js new file mode 100644 index 00000000..a469d32f --- /dev/null +++ b/src/domain/services/RepositoryDoctor.js @@ -0,0 +1,649 @@ +import { ErrorCodes } from '../errors/index.js'; +import createCasError from '../errors/createCasError.js'; +import { CACHE_SET_REF_PREFIX } from '../value-objects/CacheSetRef.js'; +import { EXPIRING_SET_REF_PREFIX } from '../value-objects/ExpiringSetRef.js'; +import { ROOT_SET_REF_PREFIX } from '../value-objects/RootSetRef.js'; +import { VAULT_REF } from './VaultPersistence.js'; + +export const DEFAULT_REPOSITORY_GRACE_PERIOD_MS = 14 * 24 * 60 * 60 * 1_000; +const DEFAULT_MAX_COLLECTIONS_PER_KIND = 100; +const MAX_COLLECTIONS_PER_KIND = 1_000; + +/** Composes bounded, non-mutating repository and managed-collection evidence. */ +export default class RepositoryDoctor { + constructor({ repository, rootSets, caches, expiringSets, vault, clock }) { + assertDependencies({ repository, rootSets, caches, expiringSets, vault }); + this.repository = repository; + this.rootSets = rootSets; + this.caches = caches; + this.expiringSets = expiringSets; + this.vault = vault; + this.clock = clock ?? { now: () => new Date() }; + Object.freeze(this); + } + + async doctor(options = {}) { + const observedAt = this.#now(); + const policy = inspectionPolicy(options, observedAt); + const limitations = baseLimitations(); + const { refs, usage } = await this.#inventoryUsage(policy.maxCollectionsPerKind, limitations); + const objects = await this.#inventoryObjects(policy.expiresBefore); + + if (!objects.consistent) { + limitations.push( + limitation( + 'REPOSITORY_CHANGED_DURING_INSPECTION', + 'Object counts changed between streamed Git inventories; derived counts are unknown.' + ) + ); + } + addTruncationLimitations(refs, limitations); + + const healthy = objects.consistent && usageHealthy(usage); + return deepFreeze({ + version: 1, + healthy, + observedAt, + completedAt: this.#now(), + policy: { + gracePeriodMs: policy.gracePeriodMs, + expiresBefore: policy.expiresBefore, + maxCollectionsPerKind: policy.maxCollectionsPerKind, + }, + repository: { + objects: objects.report, + roots: { + refCount: refs.refCount, + reflogsIncluded: true, + reflogCount: null, + }, + evidence: { + anchoredInventory: 'refs-and-reflogs', + prunableInspection: 'dry-run', + mutatesRepository: false, + }, + }, + usage, + limitations, + }); + } + + async #inventoryObjects(expiresBefore) { + const total = { objectCount: 0, logicalBytes: 0, physicalBytes: 0 }; + for await (const object of this.repository.iterateObjects()) { + total.objectCount += 1; + total.logicalBytes += object.logicalBytes; + total.physicalBytes += object.physicalBytes; + assertSafeTotals(total); + } + + const anchoredCount = await count(this.repository.iterateReachableObjectIds()); + const volatileCount = await count(this.repository.iteratePrunableObjects({ expiresBefore })); + const anchoredPhysicalBytes = await this.repository.reachablePhysicalBytes(); + const unreachableCount = total.objectCount - anchoredCount; + const orphanedCount = unreachableCount - volatileCount; + const unreachablePhysicalBytes = total.physicalBytes - anchoredPhysicalBytes; + const consistent = [unreachableCount, orphanedCount, unreachablePhysicalBytes].every( + (value) => Number.isSafeInteger(value) && value >= 0 + ); + + return { + consistent, + report: { + total, + anchored: { objectCount: anchoredCount, physicalBytes: anchoredPhysicalBytes }, + orphaned: { + objectCount: consistent ? orphanedCount : null, + physicalBytes: null, + }, + volatile: { objectCount: volatileCount, physicalBytes: null }, + unreachable: { + objectCount: consistent ? unreachableCount : null, + physicalBytes: consistent ? unreachablePhysicalBytes : null, + }, + }, + }; + } + + async #inventoryUsage(limit, limitations) { + const result = usageInventory(limit); + for await (const record of this.repository.iterateRefs()) { + result.refCount += 1; + if (record.ref.startsWith(CACHE_SET_REF_PREFIX)) { + result.caches.observed += 1; + recordCache(result.caches, await this.#inspectCache(record)); + } else if (record.ref.startsWith(ROOT_SET_REF_PREFIX)) { + result.rootSets.observed += 1; + recordRootSet(result.rootSets, await this.#inspectRootSet(record)); + } else if (record.ref.startsWith(EXPIRING_SET_REF_PREFIX)) { + result.expiringSets.observed += 1; + recordExpiringSet(result.expiringSets, await this.#inspectExpiringSet(record)); + } else if (record.ref === VAULT_REF) { + result.vault = record; + } + } + return { + refs: result, + usage: { + caches: collectionGroup(result.caches), + rootSets: collectionGroup(result.rootSets), + expiringSets: collectionGroup(result.expiringSets), + vault: await this.#inspectVault(result.vault, limitations), + }, + }; + } + + async #inspectCache(record) { + const namespace = record.ref.slice(CACHE_SET_REF_PREFIX.length); + try { + const cache = await this.caches.open({ namespace }); + return cacheUsage(record, namespace, await cache.doctor()); + } catch (error) { + return unhealthyCacheUsage({ record, namespace, error }); + } + } + + async #inspectRootSet(record) { + try { + const rootSet = await this.rootSets.open({ ref: record.ref }); + return rootSetUsage(record, await rootSet.doctor()); + } catch (error) { + return unhealthyRootSetUsage({ record, error }); + } + } + + async #inspectExpiringSet(record) { + const namespace = record.ref.slice(EXPIRING_SET_REF_PREFIX.length); + try { + const expiringSet = await this.expiringSets.open({ namespace }); + return expiringSetUsage(record, namespace, await expiringSet.doctor()); + } catch (error) { + return unhealthyExpiringSetUsage({ record, namespace, error }); + } + } + + async #inspectVault(record, limitations) { + if (record === null) { + return { + ref: VAULT_REF, + present: false, + healthy: true, + generation: null, + entryCount: 0, + physicalBytes: null, + privacy: null, + reachability: null, + issues: [], + }; + } + try { + const metadata = await this.vault.getVaultMetadata(); + const privacy = Boolean(metadata?.privacy?.enabled); + if (privacy) { + limitations.push( + limitation( + 'VAULT_ENTRY_COUNT_REQUIRES_KEY', + 'Privacy-mode vault entry count is unknown without caller-provided key material.' + ) + ); + return healthyVault(record, { entryCount: null, privacy }); + } + const state = await this.vault.readState(); + return healthyVault(record, { entryCount: state.entries.size, privacy }); + } catch (error) { + return { + ref: VAULT_REF, + present: true, + healthy: false, + generation: record.oid, + entryCount: null, + physicalBytes: null, + privacy: null, + reachability: 'anchored', + issues: [publicError(error)], + }; + } + } + + #now() { + const value = this.clock.now(); + if (!(value instanceof Date) || Number.isNaN(value.getTime())) { + throw createCasError( + 'Repository doctor clock must return a valid Date', + ErrorCodes.REPOSITORY_INSPECTION_INVALID + ); + } + return value.toISOString(); + } +} + +function inspectionPolicy(options, observedAt) { + assertOptionsObject(options); + const maxCollectionsPerKind = collectionLimit(options.maxCollectionsPerKind); + if (options.expiresBefore !== undefined) { + return explicitExpiryPolicy(options, maxCollectionsPerKind); + } + return gracePeriodPolicy(options, observedAt, maxCollectionsPerKind); +} + +function assertOptionsObject(options) { + if (!options || typeof options !== 'object' || Array.isArray(options)) { + throw invalidOptions('Repository doctor options must be an object'); + } +} + +function collectionLimit(value = DEFAULT_MAX_COLLECTIONS_PER_KIND) { + if (!Number.isSafeInteger(value) || value < 1 || value > MAX_COLLECTIONS_PER_KIND) { + throw invalidOptions( + `maxCollectionsPerKind must be an integer from 1 to ${MAX_COLLECTIONS_PER_KIND}` + ); + } + return value; +} + +function explicitExpiryPolicy(options, maxCollectionsPerKind) { + if (options.gracePeriodMs !== undefined) { + throw invalidOptions('expiresBefore and gracePeriodMs are mutually exclusive'); + } + const parsed = new Date(options.expiresBefore); + if ( + typeof options.expiresBefore !== 'string' || + Number.isNaN(parsed.getTime()) || + parsed.toISOString() !== options.expiresBefore + ) { + throw invalidOptions('expiresBefore must be a canonical millisecond UTC timestamp'); + } + return { expiresBefore: options.expiresBefore, gracePeriodMs: null, maxCollectionsPerKind }; +} + +function gracePeriodPolicy(options, observedAt, maxCollectionsPerKind) { + const gracePeriodMs = options.gracePeriodMs ?? DEFAULT_REPOSITORY_GRACE_PERIOD_MS; + if (!Number.isSafeInteger(gracePeriodMs) || gracePeriodMs < 0) { + throw invalidOptions('gracePeriodMs must be a non-negative safe integer'); + } + const expiresAt = new Date(Date.parse(observedAt) - gracePeriodMs); + if (Number.isNaN(expiresAt.getTime())) { + throw invalidOptions('gracePeriodMs exceeds the supported timestamp range'); + } + return { + expiresBefore: expiresAt.toISOString(), + gracePeriodMs, + maxCollectionsPerKind, + }; +} + +function cacheUsage(record, namespace, report) { + if (!report.state) { + return cacheUsageWithoutState(record, namespace, report); + } + const { state } = report; + const policy = report.policy ? report.policy.limits : state.policy; + return { + namespace, + ref: record.ref, + generation: rootGeneration(report, record.oid), + healthy: report.healthy, + entryCount: state.entryCount, + logicalBytes: state.logicalBytes, + physicalBytes: null, + retention: { + pinnedEntries: state.pinnedEntries, + evictableEntries: state.evictableEntries, + }, + reachability: 'anchored', + age: { + createdAt: state.createdAt, + updatedAt: state.updatedAt, + oldestAccessedAt: state.oldestAccessedAt, + }, + expiry: { + expiredEntries: state.expiredEntries, + nextExpiry: state.nextExpiry, + }, + policy, + issues: report.issues ?? [], + }; +} + +function cacheUsageWithoutState(record, namespace, report) { + const knownEmpty = report.healthy; + return { + namespace, + ref: record.ref, + generation: rootGeneration(report, record.oid), + healthy: report.healthy, + entryCount: knownEmpty ? 0 : null, + logicalBytes: knownEmpty ? 0 : null, + physicalBytes: null, + retention: knownEmpty ? { pinnedEntries: 0, evictableEntries: 0 } : null, + reachability: 'anchored', + age: knownEmpty ? { createdAt: null, updatedAt: null, oldestAccessedAt: null } : null, + expiry: knownEmpty ? { expiredEntries: 0, nextExpiry: null } : null, + policy: null, + issues: report.issues ?? [], + }; +} + +function rootSetUsage(record, report) { + const counts = report.policyCounts; + return { + ref: record.ref, + generation: report.headOid ?? record.oid, + healthy: report.healthy, + entryCount: report.entryCount ?? null, + physicalBytes: null, + retention: counts ? { pinnedEntries: counts.pinned, evictableEntries: counts.evictable } : null, + reachability: report.reachabilityCounts ?? null, + issues: report.issues ?? errorIssues(report), + }; +} + +function expiringSetUsage(record, namespace, report) { + if (!report.state) { + return expiringSetUsageWithoutState(record, namespace, report); + } + return { + namespace, + ref: record.ref, + generation: rootGeneration(report, record.oid), + healthy: report.healthy, + entryCount: report.state.entryCount, + physicalBytes: null, + reachability: 'anchored', + age: { + createdAt: report.state.createdAt, + updatedAt: report.state.updatedAt, + }, + expiry: { + liveEntries: report.observed.liveEntries, + expiredEntries: report.observed.expiredEntries, + nextExpiry: report.state.nextExpiry, + }, + issues: report.issues ?? [], + }; +} + +function expiringSetUsageWithoutState(record, namespace, report) { + const knownEmpty = report.healthy; + return { + namespace, + ref: record.ref, + generation: rootGeneration(report, record.oid), + healthy: report.healthy, + entryCount: knownEmpty ? 0 : null, + physicalBytes: null, + reachability: 'anchored', + age: knownEmpty ? { createdAt: null, updatedAt: null } : null, + expiry: knownEmpty ? { liveEntries: 0, expiredEntries: 0, nextExpiry: null } : null, + issues: report.issues ?? [], + }; +} + +function rootGeneration(report, fallback) { + return report.root?.headOid ?? fallback; +} + +function errorIssues(report) { + return report.error ? [report.error] : []; +} + +async function count(iterable) { + let result = 0; + for await (const value of iterable) { + void value; + result += 1; + } + return result; +} + +function collectionInventory(totals, detailLimit) { + return { observed: 0, inspected: 0, healthy: true, totals, entries: [], detailLimit }; +} + +function usageInventory(detailLimit) { + return { + refCount: 0, + caches: collectionInventory( + { + entryCount: 0, + logicalBytes: 0, + pinnedEntries: 0, + evictableEntries: 0, + expiredEntries: 0, + }, + detailLimit + ), + rootSets: collectionInventory( + { entryCount: 0, pinnedEntries: 0, evictableEntries: 0 }, + detailLimit + ), + expiringSets: collectionInventory( + { entryCount: 0, liveEntries: 0, expiredEntries: 0 }, + detailLimit + ), + vault: null, + }; +} + +function coverage(inventory) { + return { + observed: inventory.observed, + inspected: inventory.inspected, + detailed: inventory.entries.length, + complete: inventory.observed === inventory.inspected, + }; +} + +function recordCache(inventory, entry) { + recordCollection(inventory, entry, { + entryCount: entry.entryCount, + logicalBytes: entry.logicalBytes, + pinnedEntries: entry.retention?.pinnedEntries, + evictableEntries: entry.retention?.evictableEntries, + expiredEntries: entry.expiry?.expiredEntries, + }); +} + +function recordRootSet(inventory, entry) { + recordCollection(inventory, entry, { + entryCount: entry.entryCount, + pinnedEntries: entry.retention?.pinnedEntries, + evictableEntries: entry.retention?.evictableEntries, + }); +} + +function recordExpiringSet(inventory, entry) { + recordCollection(inventory, entry, { + entryCount: entry.entryCount, + liveEntries: entry.expiry?.liveEntries, + expiredEntries: entry.expiry?.expiredEntries, + }); +} + +function recordCollection(inventory, entry, values) { + inventory.inspected += 1; + inventory.healthy &&= entry.healthy; + for (const [key, value] of Object.entries(values)) { + addKnown(inventory.totals, key, value); + } + if (inventory.entries.length < inventory.detailLimit) { + inventory.entries.push(entry); + } +} + +function collectionGroup(inventory) { + return { + healthy: inventory.healthy, + coverage: coverage(inventory), + totals: inventory.totals, + entries: inventory.entries, + }; +} + +function addKnown(totals, key, value) { + if (totals[key] === null) { + return; + } + const next = totals[key] + value; + totals[key] = Number.isSafeInteger(value) && Number.isSafeInteger(next) ? next : null; +} + +function healthyVault(record, { entryCount, privacy }) { + return { + ref: VAULT_REF, + present: true, + healthy: true, + generation: record.oid, + entryCount, + physicalBytes: null, + privacy, + reachability: 'anchored', + issues: [], + }; +} + +function usageHealthy(usage) { + return ( + usage.caches.healthy && + usage.rootSets.healthy && + usage.expiringSets.healthy && + usage.vault.healthy + ); +} + +function unhealthyBase({ record, namespace, error }) { + return { + ...(namespace === undefined ? {} : { namespace }), + ref: record.ref, + generation: record.oid, + healthy: false, + entryCount: null, + physicalBytes: null, + reachability: 'anchored', + issues: [publicError(error)], + }; +} + +function unhealthyCacheUsage(options) { + return { + ...unhealthyBase(options), + logicalBytes: null, + retention: null, + age: null, + expiry: null, + policy: null, + }; +} + +function unhealthyRootSetUsage(options) { + return { ...unhealthyBase(options), retention: null }; +} + +function unhealthyExpiringSetUsage(options) { + return { ...unhealthyBase(options), age: null, expiry: null }; +} + +function publicError(error) { + return { + code: error?.code ?? ErrorCodes.REPOSITORY_INSPECTION_INVALID, + message: error instanceof Error ? error.message : String(error), + }; +} + +function baseLimitations() { + return [ + limitation( + 'SHARED_PHYSICAL_BYTES_UNATTRIBUTABLE', + 'Deduplicated Git objects cannot be assigned exactly to one cache, root set, or vault.' + ), + limitation( + 'PACKED_OBJECT_AGE_UNAVAILABLE', + 'Git prune dry-run reports loose candidates; packed unreachable object age is not observable.' + ), + limitation( + 'REFLOG_COUNT_UNAVAILABLE', + 'Reflogs participate in reachability, but this bounded report does not enumerate reflog entries.' + ), + limitation( + 'ALTERNATE_OBJECT_STORES_INCLUDED', + 'Git may include objects and bytes from configured alternate object stores.' + ), + limitation( + 'PACK_OVERHEAD_EXCLUDED', + 'Git object disk sizes exclude pack indexes, bitmaps, and other repository metadata.' + ), + ]; +} + +function addTruncationLimitations(refs, limitations) { + for (const [kind, inventory] of [ + ['caches', refs.caches], + ['rootSets', refs.rootSets], + ['expiringSets', refs.expiringSets], + ]) { + if (inventory.observed > inventory.entries.length) { + limitations.push({ + ...limitation( + 'COLLECTION_DETAILS_TRUNCATED', + 'Managed collection detail exceeded maxCollectionsPerKind.' + ), + kind, + observed: inventory.observed, + inspected: inventory.inspected, + detailed: inventory.entries.length, + }); + } + } +} + +function limitation(code, message) { + return { code, message }; +} + +function assertSafeTotals(total) { + if (!Object.values(total).every(Number.isSafeInteger)) { + throw invalidOptions('Repository object totals exceed JavaScript safe integer bounds'); + } +} + +function assertDependencies(value) { + const checks = [ + [ + value.repository, + [ + 'iterateObjects', + 'iterateReachableObjectIds', + 'iteratePrunableObjects', + 'iterateRefs', + 'reachablePhysicalBytes', + ], + ], + [value.rootSets, ['open']], + [value.caches, ['open']], + [value.expiringSets, ['open']], + [value.vault, ['getVaultMetadata', 'readState']], + ]; + if ( + checks.some(([target, methods]) => + methods.some((method) => typeof target?.[method] !== 'function') + ) + ) { + throw invalidOptions( + 'RepositoryDoctor requires complete repository and collection dependencies' + ); + } +} + +function invalidOptions(message) { + return createCasError(message, ErrorCodes.REPOSITORY_INSPECTION_INVALID); +} + +function deepFreeze(value) { + if (!value || typeof value !== 'object' || Object.isFrozen(value)) { + return value; + } + for (const child of Object.values(value)) { + deepFreeze(child); + } + return Object.freeze(value); +} diff --git a/src/infrastructure/adapters/GitRepositoryInspectionAdapter.js b/src/infrastructure/adapters/GitRepositoryInspectionAdapter.js new file mode 100644 index 00000000..3a96667d --- /dev/null +++ b/src/infrastructure/adapters/GitRepositoryInspectionAdapter.js @@ -0,0 +1,164 @@ +import { Policy } from '@git-stunts/alfred'; +import { CasError, ErrorCodes } from '../../domain/errors/index.js'; +import Oid from '../../domain/value-objects/Oid.js'; +import RepositoryInspectionPort from '../../ports/RepositoryInspectionPort.js'; + +const DEFAULT_POLICY = Policy.timeout(30_000); +const DEFAULT_FULL_SCAN_POLICY = Policy.timeout(5 * 60_000); +const OBJECT_TYPES = new Set(['blob', 'tree', 'commit', 'tag']); +const OBJECT_FORMAT = '--batch-check=%(objectname) %(objecttype) %(objectsize) %(objectsize:disk)'; + +/** Non-mutating repository inspection backed by safe Git plumbing commands. */ +export default class GitRepositoryInspectionAdapter extends RepositoryInspectionPort { + constructor({ plumbing, policy }) { + super(); + if ( + typeof plumbing?.execute !== 'function' || + typeof plumbing?.executeStream !== 'function' || + typeof plumbing?.inspectPrunableObjects !== 'function' + ) { + throw new CasError( + 'Repository inspection requires GitPlumbing 3.1.0 or newer', + ErrorCodes.REPOSITORY_INSPECTION_INVALID + ); + } + this.plumbing = plumbing; + this.policy = policy ?? DEFAULT_POLICY; + this.fullScanPolicy = policy ?? DEFAULT_FULL_SCAN_POLICY; + Object.freeze(this); + } + + async *iterateObjects() { + const stream = await this.#stream({ + args: ['cat-file', '--batch-all-objects', OBJECT_FORMAT], + }); + for await (const line of consumeLines(stream, 'object inventory')) { + const fields = line.split(' '); + if (fields.length !== 4) { + throw invalidOutput('object inventory', line); + } + const [oid, type] = fields; + yield Object.freeze({ + oid: parseOid(oid, 'object inventory', line), + type: parseType(type, 'object inventory', line), + logicalBytes: parseBytes(fields[2], 'object inventory', line), + physicalBytes: parseBytes(fields[3], 'object inventory', line), + }); + } + } + + async *iterateReachableObjectIds() { + const stream = await this.#stream({ + args: ['rev-list', '--all', '--reflog', '--objects', '--no-object-names'], + }); + for await (const line of consumeLines(stream, 'reachable object inventory')) { + yield parseOid(line, 'reachable object inventory', line); + } + } + + async *iteratePrunableObjects({ expiresBefore }) { + const stream = await this.policy.execute(() => + this.plumbing.inspectPrunableObjects({ expiresBefore }) + ); + for await (const line of consumeLines(stream, 'prunable object inspection')) { + const fields = line.trim().split(/\s+/u); + if (fields.length !== 2) { + throw invalidOutput('prunable object inspection', line); + } + yield Object.freeze({ + oid: parseOid(fields[0], 'prunable object inspection', line), + type: parseType(fields[1], 'prunable object inspection', line), + }); + } + } + + async *iterateRefs() { + const stream = await this.#stream({ + args: ['for-each-ref', '--format=%(refname)%09%(objectname)', 'refs/'], + }); + for await (const line of consumeLines(stream, 'ref inventory')) { + const fields = line.split('\t'); + if (fields.length !== 2 || !fields[0].startsWith('refs/')) { + throw invalidOutput('ref inventory', line); + } + yield Object.freeze({ + ref: fields[0], + oid: parseOid(fields[1], 'ref inventory', line), + }); + } + } + + async reachablePhysicalBytes() { + const output = await this.fullScanPolicy.execute(() => + this.plumbing.execute({ + args: ['rev-list', '--all', '--reflog', '--objects', '--disk-usage'], + }) + ); + return parseBytes(String(output).trim(), 'reachable disk usage', output); + } + + async #stream(options) { + return await this.policy.execute(() => this.plumbing.executeStream(options)); + } +} + +async function* consumeLines(stream, operation) { + const decoder = new globalThis.TextDecoder(); + let pending = ''; + for await (const chunk of stream) { + pending += typeof chunk === 'string' ? chunk : decoder.decode(chunk, { stream: true }); + let newline = pending.indexOf('\n'); + while (newline !== -1) { + const line = pending.slice(0, newline).replace(/\r$/u, ''); + pending = pending.slice(newline + 1); + if (line.length > 0) { + yield line; + } + newline = pending.indexOf('\n'); + } + } + pending += decoder.decode(); + if (pending.length > 0) { + yield pending.replace(/\r$/u, ''); + } + const result = await stream.finished; + if (result?.code !== 0) { + throw new CasError(`Git ${operation} failed`, ErrorCodes.REPOSITORY_INSPECTION_INVALID, { + operation, + stderr: result?.stderr, + }); + } +} + +function parseOid(value, operation, output) { + if (!Oid.isValid(value)) { + throw invalidOutput(operation, output); + } + return Oid.from(value).toString(); +} + +function parseType(value, operation, output) { + if (!OBJECT_TYPES.has(value)) { + throw invalidOutput(operation, output); + } + return value; +} + +function parseBytes(value, operation, output) { + if (!/^\d+$/u.test(String(value))) { + throw invalidOutput(operation, output); + } + const bytes = Number(value); + if (!Number.isSafeInteger(bytes)) { + throw invalidOutput(operation, output); + } + return bytes; +} + +function invalidOutput(operation, output) { + return new CasError( + `Git ${operation} returned invalid structured output`, + ErrorCodes.REPOSITORY_INSPECTION_INVALID, + { operation, output: String(output) } + ); +} diff --git a/src/ports/RepositoryInspectionPort.js b/src/ports/RepositoryInspectionPort.js new file mode 100644 index 00000000..bbaeadee --- /dev/null +++ b/src/ports/RepositoryInspectionPort.js @@ -0,0 +1,33 @@ +/** + * Abstract port for non-mutating, repository-wide Git object inspection. + * @abstract + */ +export default class RepositoryInspectionPort { + /** @returns {AsyncIterable<{ oid: string, type: string, logicalBytes: number, physicalBytes: number }>} */ + iterateObjects() { + throw new Error('Not implemented'); + } + + /** @returns {AsyncIterable} */ + iterateReachableObjectIds() { + throw new Error('Not implemented'); + } + + /** + * @param {{ expiresBefore: string }} _options + * @returns {AsyncIterable<{ oid: string, type: string }>} + */ + iteratePrunableObjects(_options) { + throw new Error('Not implemented'); + } + + /** @returns {AsyncIterable<{ ref: string, oid: string }>} */ + iterateRefs() { + throw new Error('Not implemented'); + } + + /** @returns {Promise} */ + async reachablePhysicalBytes() { + throw new Error('Not implemented'); + } +} diff --git a/test/integration/repository-diagnostics.test.js b/test/integration/repository-diagnostics.test.js new file mode 100644 index 00000000..7c165be9 --- /dev/null +++ b/test/integration/repository-diagnostics.test.js @@ -0,0 +1,169 @@ +/** Real-Git proof for bounded, non-mutating repository-wide diagnostics. */ + +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; +import { mkdtempSync, rmSync, utimesSync } from 'node:fs'; +import { spawnSync } from 'node:child_process'; +import os from 'node:os'; +import path from 'node:path'; +import ContentAddressableStore from '../../index.js'; +import { createGitPlumbing } from '../../src/infrastructure/createGitPlumbing.js'; + +if (process.env.GIT_STUNTS_DOCKER !== '1') { + throw new Error( + 'Integration tests MUST run inside Docker (GIT_STUNTS_DOCKER=1). ' + + 'Use: npm run test:integration:node' + ); +} + +vi.setConfig({ testTimeout: 30_000, hookTimeout: 30_000 }); + +const EXPIRES_BEFORE = '2026-07-01T00:00:00.000Z'; +let cas; +let oldLooseOid; +let recentLooseOid; +let reflogCommitOid; +let repoDir; + +function git(args, input) { + const result = spawnSync('git', args, { cwd: repoDir, encoding: 'utf8', input }); + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + throw new Error(`${result.stderr || result.stdout || 'git failed'}`.trim()); + } + return result.stdout.trim(); +} + +function objectIds() { + return git(['cat-file', '--batch-all-objects', '--batch-check=%(objectname)']) + .split('\n') + .filter(Boolean); +} + +function refs() { + return git(['for-each-ref', '--format=%(refname)%09%(objectname)', 'refs/']) + .split('\n') + .filter(Boolean); +} + +function createCommit(content, message) { + const blob = git(['hash-object', '-w', '--stdin'], content); + const tree = git(['mktree'], `100644 blob ${blob}\tpayload.txt\n`); + const commit = git(['commit-tree', tree, '-m', message]); + return { blob, tree, commit }; +} + +function makeOld(oid) { + const objectPath = path.join(repoDir, 'objects', oid.slice(0, 2), oid.slice(2)); + const old = new Date('2026-06-01T00:00:00.000Z'); + utimesSync(objectPath, old, old); +} + +async function* bytes(value) { + yield Buffer.from(value); +} + +beforeAll(async () => { + repoDir = mkdtempSync(path.join(os.tmpdir(), 'cas-repository-doctor-')); + git(['init', '--bare']); + git(['config', 'user.name', 'Repository Doctor']); + git(['config', 'user.email', 'doctor@example.test']); + git(['config', 'core.logAllRefUpdates', 'true']); + + const reflogRoot = createCommit('reflog-only', 'reflog root'); + const currentRoot = createCommit('current-branch', 'current root'); + reflogCommitOid = reflogRoot.commit; + git(['update-ref', '--create-reflog', 'refs/heads/main', reflogRoot.commit]); + git(['update-ref', 'refs/heads/main', currentRoot.commit, reflogRoot.commit]); + git(['tag', '-a', 'v1', '-m', 'diagnostic tag', currentRoot.commit]); + + cas = new ContentAddressableStore({ + plumbing: await createGitPlumbing({ cwd: repoDir }), + clock: { now: () => new Date('2026-07-13T12:00:00.000Z') }, + }); + const rootPage = await cas.pages.put({ source: Buffer.from('root-set target') }); + const rootSet = await cas.rootSets.open({ ref: 'refs/cas/rootsets/git-warp/live' }); + await rootSet.put({ + name: 'live', + oid: rootPage.handle.oid, + type: 'blob', + retention: 'pinned', + }); + const cachePage = await cas.pages.put({ source: Buffer.from('cache target') }); + const cache = await cas.caches.open({ namespace: 'git-warp/materializations' }); + await cache.put('coordinate:1', cachePage.handle, { retention: 'evictable' }); + const expiring = await cas.expiringSets.open({ namespace: 'git-warp/replay' }); + await expiring.addIfAbsent('request:1', { expiresAt: '2026-07-20T00:00:00.000Z' }); + const asset = await cas.assets.put({ + source: bytes('vault target'), + slug: 'diagnostics/vault-target', + filename: 'vault.txt', + }); + await cas.initVault(); + await cas.addToVault({ slug: 'diagnostics/vault-target', treeOid: asset.handle.oid }); + + recentLooseOid = git(['hash-object', '-w', '--stdin'], 'recent-unreachable'); + oldLooseOid = git(['hash-object', '-w', '--stdin'], 'past-grace-unreachable'); + makeOld(oldLooseOid); +}); + +afterAll(() => { + rmSync(repoDir, { recursive: true, force: true }); +}); + +// eslint-disable-next-line max-lines-per-function +describe('repository diagnostics', () => { + // eslint-disable-next-line max-lines-per-function + it('classifies refs, reflogs, recent orphans, and volatile loose objects without mutation', async () => { + const objectsBefore = objectIds(); + const refsBefore = refs(); + const reachable = new Set( + git(['rev-list', '--all', '--reflog', '--objects', '--no-object-names']).split('\n') + ); + const prunable = new Set( + git(['prune', '--dry-run', '--verbose', '--no-progress', `--expire=${EXPIRES_BEFORE}`]) + .split('\n') + .filter(Boolean) + .map((line) => line.split(' ')[0]) + ); + + expect(reachable).toContain(reflogCommitOid); + expect(reachable).not.toContain(recentLooseOid); + expect(reachable).not.toContain(oldLooseOid); + expect(prunable).not.toContain(recentLooseOid); + expect(prunable).toContain(oldLooseOid); + + const report = await cas.diagnostics.doctor({ expiresBefore: EXPIRES_BEFORE }); + + expect(report.healthy).toBe(true); + expect(report.repository.objects.volatile.objectCount).toBeGreaterThanOrEqual(1); + expect(report.repository.objects.orphaned.objectCount).toBeGreaterThanOrEqual(1); + expect(report.repository.objects.total.objectCount).toBe( + report.repository.objects.anchored.objectCount + + report.repository.objects.orphaned.objectCount + + report.repository.objects.volatile.objectCount + ); + expect(report.repository.roots).toMatchObject({ reflogsIncluded: true }); + expect(report.usage.caches).toMatchObject({ + coverage: { observed: 1, inspected: 1, complete: true }, + totals: { entryCount: 1 }, + }); + expect(report.usage.rootSets).toMatchObject({ + coverage: { observed: 1, inspected: 1, complete: true }, + totals: { entryCount: 1, pinnedEntries: 1 }, + }); + expect(report.usage.expiringSets).toMatchObject({ + coverage: { observed: 1, inspected: 1, complete: true }, + totals: { entryCount: 1, liveEntries: 1 }, + }); + expect(report.usage.vault).toMatchObject({ + present: true, + healthy: true, + entryCount: 1, + reachability: 'anchored', + }); + expect(objectIds()).toEqual(objectsBefore); + expect(refs()).toEqual(refsBefore); + }); +}); diff --git a/test/unit/domain/services/RepositoryDoctor.test.js b/test/unit/domain/services/RepositoryDoctor.test.js new file mode 100644 index 00000000..96056ad4 --- /dev/null +++ b/test/unit/domain/services/RepositoryDoctor.test.js @@ -0,0 +1,335 @@ +import { describe, expect, it, vi } from 'vitest'; +import RepositoryDoctor, { + DEFAULT_REPOSITORY_GRACE_PERIOD_MS, +} from '../../../../src/domain/services/RepositoryDoctor.js'; + +const NOW = '2026-07-13T12:00:00.000Z'; + +async function* values(items) { + yield* items; +} + +function repository(overrides = {}) { + return { + iterateObjects: vi.fn(() => + values([ + { oid: '1'.repeat(40), type: 'blob', logicalBytes: 10, physicalBytes: 8 }, + { oid: '2'.repeat(40), type: 'tree', logicalBytes: 20, physicalBytes: 12 }, + { oid: '3'.repeat(40), type: 'commit', logicalBytes: 30, physicalBytes: 16 }, + { oid: '4'.repeat(40), type: 'blob', logicalBytes: 40, physicalBytes: 20 }, + { oid: '5'.repeat(40), type: 'blob', logicalBytes: 50, physicalBytes: 24 }, + ]) + ), + iterateReachableObjectIds: vi.fn(() => + values(['1'.repeat(40), '2'.repeat(40), '3'.repeat(40)]) + ), + iteratePrunableObjects: vi.fn(() => values([{ oid: '5'.repeat(40), type: 'blob' }])), + iterateRefs: vi.fn(() => + values([ + { ref: 'refs/cas/caches/git-warp/materializations', oid: 'a'.repeat(40) }, + { ref: 'refs/cas/expiring/git-warp/replay', oid: 'b'.repeat(40) }, + { ref: 'refs/cas/rootsets/git-warp/live', oid: 'c'.repeat(40) }, + { ref: 'refs/cas/vault', oid: 'd'.repeat(40) }, + { ref: 'refs/heads/main', oid: 'e'.repeat(40) }, + { ref: 'refs/tags/v1', oid: 'f'.repeat(40) }, + ]) + ), + reachablePhysicalBytes: vi.fn().mockResolvedValue(36), + ...overrides, + }; +} + +function cacheDoctor() { + return { + healthy: true, + root: { headOid: 'a'.repeat(40) }, + state: { + entryCount: 2, + logicalBytes: 90, + pinnedEntries: 1, + evictableEntries: 1, + expiredEntries: 1, + createdAt: '2026-07-01T00:00:00.000Z', + updatedAt: '2026-07-12T00:00:00.000Z', + oldestAccessedAt: '2026-07-02T00:00:00.000Z', + nextExpiry: '2026-07-14T00:00:00.000Z', + }, + policy: { + satisfied: true, + limits: { maxEntries: 10, maxBytes: 1_000, accessResolutionMs: 60_000 }, + }, + issues: [], + }; +} + +function dependencies(repositoryPort = repository()) { + return { + repository: repositoryPort, + rootSets: { + open: vi.fn(() => ({ + doctor: vi.fn().mockResolvedValue({ + healthy: true, + headOid: 'c'.repeat(40), + entryCount: 3, + policyCounts: { pinned: 2, evictable: 1 }, + reachabilityCounts: { anchored: 3, missing: 0, unknown: 0, orphaned: 0, volatile: 0 }, + issues: [], + }), + })), + }, + caches: { + open: vi.fn(() => ({ doctor: vi.fn().mockResolvedValue(cacheDoctor()) })), + }, + expiringSets: { + open: vi.fn(() => ({ + doctor: vi.fn().mockResolvedValue({ + healthy: true, + root: { headOid: 'b'.repeat(40) }, + state: { + entryCount: 4, + createdAt: '2026-07-01T00:00:00.000Z', + updatedAt: '2026-07-12T00:00:00.000Z', + nextExpiry: '2026-07-20T00:00:00.000Z', + }, + observed: { liveEntries: 3, expiredEntries: 1 }, + issues: [], + }), + })), + }, + vault: { + getVaultMetadata: vi.fn().mockResolvedValue({ version: 1 }), + readState: vi.fn().mockResolvedValue({ + entries: new Map([ + ['one', '1'.repeat(40)], + ['two', '2'.repeat(40)], + ]), + parentCommitOid: 'd'.repeat(40), + metadata: { version: 1 }, + }), + }, + clock: { now: () => new Date(NOW) }, + }; +} + +// eslint-disable-next-line max-lines-per-function +describe('RepositoryDoctor', () => { + // eslint-disable-next-line max-lines-per-function + it('keeps reachability, retention policy, and byte attribution as separate evidence', async () => { + const ports = dependencies(); + const doctor = new RepositoryDoctor(ports); + const report = await doctor.doctor(); + + expect(report).toMatchObject({ + version: 1, + healthy: true, + observedAt: NOW, + policy: { + gracePeriodMs: DEFAULT_REPOSITORY_GRACE_PERIOD_MS, + expiresBefore: '2026-06-29T12:00:00.000Z', + }, + repository: { + objects: { + total: { objectCount: 5, logicalBytes: 150, physicalBytes: 80 }, + anchored: { objectCount: 3, physicalBytes: 36 }, + orphaned: { objectCount: 1, physicalBytes: null }, + volatile: { objectCount: 1, physicalBytes: null }, + unreachable: { objectCount: 2, physicalBytes: 44 }, + }, + roots: { refCount: 6, reflogsIncluded: true }, + evidence: { prunableInspection: 'dry-run', mutatesRepository: false }, + }, + usage: { + caches: { + coverage: { observed: 1, inspected: 1, complete: true }, + totals: { + entryCount: 2, + logicalBytes: 90, + pinnedEntries: 1, + evictableEntries: 1, + expiredEntries: 1, + }, + }, + rootSets: { + coverage: { observed: 1, inspected: 1, complete: true }, + totals: { entryCount: 3, pinnedEntries: 2, evictableEntries: 1 }, + }, + vault: { present: true, healthy: true, entryCount: 2, physicalBytes: null }, + }, + }); + expect(report.usage.caches.entries[0]).toMatchObject({ + namespace: 'git-warp/materializations', + reachability: 'anchored', + retention: { pinnedEntries: 1, evictableEntries: 1 }, + physicalBytes: null, + }); + expect(report.limitations.map(({ code }) => code)).toEqual( + expect.arrayContaining([ + 'SHARED_PHYSICAL_BYTES_UNATTRIBUTABLE', + 'PACKED_OBJECT_AGE_UNAVAILABLE', + ]) + ); + expect(Object.isFrozen(report)).toBe(true); + expect(Object.isFrozen(report.repository.objects)).toBe(true); + expect(Object.isFrozen(report.usage.caches.entries)).toBe(true); + expect(ports.repository.iteratePrunableObjects).toHaveBeenCalledWith({ + expiresBefore: '2026-06-29T12:00:00.000Z', + }); + }); + + it('bounds collection details while inspecting and totaling every managed ref', async () => { + const repositoryPort = repository({ + iterateRefs: vi.fn(() => + values([ + { ref: 'refs/cas/caches/a/one', oid: 'a'.repeat(40) }, + { ref: 'refs/cas/caches/b/two', oid: 'b'.repeat(40) }, + { ref: 'refs/cas/caches/c/three', oid: 'c'.repeat(40) }, + ]) + ), + }); + const ports = dependencies(repositoryPort); + const doctor = new RepositoryDoctor(ports); + + const report = await doctor.doctor({ maxCollectionsPerKind: 2 }); + + expect(report.usage.caches.coverage).toEqual({ + observed: 3, + inspected: 3, + detailed: 2, + complete: true, + }); + expect(report.usage.caches.totals).toMatchObject({ + entryCount: 6, + logicalBytes: 270, + }); + expect(report.usage.caches.entries).toHaveLength(2); + expect(ports.caches.open).toHaveBeenCalledTimes(3); + expect(report.limitations).toContainEqual( + expect.objectContaining({ + code: 'COLLECTION_DETAILS_TRUNCATED', + kind: 'caches', + observed: 3, + inspected: 3, + detailed: 2, + }) + ); + }); + + it('returns kind-specific public shapes when managed collection inspection fails', async () => { + const ports = dependencies(); + ports.caches.open.mockRejectedValue(new Error('cache failed')); + ports.rootSets.open.mockRejectedValue(new Error('root set failed')); + ports.expiringSets.open.mockRejectedValue(new Error('expiring set failed')); + const doctor = new RepositoryDoctor(ports); + + const report = await doctor.doctor(); + + expect(report.healthy).toBe(false); + expect(report.usage.caches.entries[0]).toMatchObject({ + namespace: 'git-warp/materializations', + healthy: false, + logicalBytes: null, + retention: null, + age: null, + expiry: null, + policy: null, + }); + expect(report.usage.rootSets.entries[0]).toMatchObject({ + healthy: false, + retention: null, + }); + expect(report.usage.rootSets.entries[0]).not.toHaveProperty('logicalBytes'); + expect(report.usage.rootSets.entries[0]).not.toHaveProperty('age'); + expect(report.usage.rootSets.entries[0]).not.toHaveProperty('expiry'); + expect(report.usage.rootSets.entries[0]).not.toHaveProperty('policy'); + expect(report.usage.expiringSets.entries[0]).toMatchObject({ + namespace: 'git-warp/replay', + healthy: false, + age: null, + expiry: null, + }); + expect(report.usage.expiringSets.entries[0]).not.toHaveProperty('logicalBytes'); + expect(report.usage.expiringSets.entries[0]).not.toHaveProperty('retention'); + expect(report.usage.expiringSets.entries[0]).not.toHaveProperty('policy'); + expect(report.usage.caches.entries[0].issues[0]).toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + message: 'cache failed', + }); + expect(report.usage.rootSets.entries[0].issues[0].message).toBe('root set failed'); + expect(report.usage.expiringSets.entries[0].issues[0].message).toBe('expiring set failed'); + }); + + it('reports private-vault entry attribution as unknown without requesting a key', async () => { + const ports = dependencies(); + ports.vault.getVaultMetadata.mockResolvedValue({ + version: 1, + privacy: { enabled: true }, + }); + const doctor = new RepositoryDoctor(ports); + + const report = await doctor.doctor(); + + expect(report.usage.vault).toMatchObject({ + present: true, + healthy: true, + entryCount: null, + privacy: true, + }); + expect(ports.vault.readState).not.toHaveBeenCalled(); + expect(report.limitations).toContainEqual( + expect.objectContaining({ + code: 'VAULT_ENTRY_COUNT_REQUIRES_KEY', + }) + ); + }); + + it('marks a non-atomic inventory as unknown when concurrent writes violate count invariants', async () => { + const repositoryPort = repository({ + iterateReachableObjectIds: vi.fn(() => + values([ + '1'.repeat(40), + '2'.repeat(40), + '3'.repeat(40), + '4'.repeat(40), + '5'.repeat(40), + '6'.repeat(40), + ]) + ), + }); + const doctor = new RepositoryDoctor(dependencies(repositoryPort)); + + const report = await doctor.doctor(); + + expect(report.healthy).toBe(false); + expect(report.repository.objects.orphaned.objectCount).toBeNull(); + expect(report.repository.objects.unreachable.objectCount).toBeNull(); + expect(report.limitations).toContainEqual( + expect.objectContaining({ + code: 'REPOSITORY_CHANGED_DURING_INSPECTION', + }) + ); + }); + + it('validates explicit expiry and collection bounds before opening Git streams', async () => { + const ports = dependencies(); + const doctor = new RepositoryDoctor(ports); + + await expect(doctor.doctor({ expiresBefore: 'yesterday' })).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + await expect(doctor.doctor({ gracePeriodMs: -1 })).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + await expect(doctor.doctor({ gracePeriodMs: Number.MAX_SAFE_INTEGER })).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + await expect( + doctor.doctor({ expiresBefore: NOW, gracePeriodMs: 1 }) + ).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + await expect(doctor.doctor({ maxCollectionsPerKind: 0 })).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + expect(ports.repository.iterateObjects).not.toHaveBeenCalled(); + }); +}); diff --git a/test/unit/facade/ContentAddressableStore.diagnostics.test.js b/test/unit/facade/ContentAddressableStore.diagnostics.test.js new file mode 100644 index 00000000..95925dbd --- /dev/null +++ b/test/unit/facade/ContentAddressableStore.diagnostics.test.js @@ -0,0 +1,48 @@ +import { describe, expect, it, vi } from 'vitest'; +import ContentAddressableStore, { + GitRepositoryInspectionAdapter, + RepositoryDoctor, + RepositoryInspectionPort, +} from '../../../index.js'; + +function emptyStream() { + return { + finished: Promise.resolve({ code: 0, stderr: '' }), + async *[Symbol.asyncIterator]() {}, + }; +} + +describe('ContentAddressableStore diagnostics', () => { + it('exposes a machine-readable, non-mutating repository doctor capability', async () => { + const plumbing = { + execute: vi.fn().mockResolvedValue('0\n'), + executeStream: vi.fn().mockImplementation(() => Promise.resolve(emptyStream())), + inspectPrunableObjects: vi.fn().mockImplementation(() => Promise.resolve(emptyStream())), + }; + const cas = new ContentAddressableStore({ + plumbing, + clock: { now: () => new Date('2026-07-13T12:00:00.000Z') }, + }); + + await expect(cas.diagnostics.doctor()).resolves.toMatchObject({ + version: 1, + healthy: true, + repository: { + objects: { + total: { objectCount: 0, logicalBytes: 0, physicalBytes: 0 }, + anchored: { objectCount: 0, physicalBytes: 0 }, + orphaned: { objectCount: 0 }, + volatile: { objectCount: 0 }, + }, + evidence: { mutatesRepository: false }, + }, + }); + expect(plumbing.inspectPrunableObjects).toHaveBeenCalledTimes(1); + }); + + it('exports the diagnostics domain and adapter boundaries', () => { + expect(RepositoryDoctor).toBeTypeOf('function'); + expect(GitRepositoryInspectionAdapter).toBeTypeOf('function'); + expect(RepositoryInspectionPort).toBeTypeOf('function'); + }); +}); diff --git a/test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js b/test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js new file mode 100644 index 00000000..4a9a8079 --- /dev/null +++ b/test/unit/infrastructure/adapters/GitRepositoryInspectionAdapter.test.js @@ -0,0 +1,172 @@ +import { describe, expect, it, vi } from 'vitest'; +import GitRepositoryInspectionAdapter from '../../../../src/infrastructure/adapters/GitRepositoryInspectionAdapter.js'; + +function gitStream(chunks, result = { code: 0, stderr: '' }) { + return { + finished: Promise.resolve(result), + async *[Symbol.asyncIterator]() { + for (const chunk of chunks) { + yield chunk; + } + }, + }; +} + +function mockPlumbing(overrides = {}) { + return { + execute: vi.fn(), + executeStream: vi.fn(), + inspectPrunableObjects: vi.fn(), + ...overrides, + }; +} + +async function collect(iterable) { + const values = []; + for await (const value of iterable) { + values.push(value); + } + return values; +} + +// eslint-disable-next-line max-lines-per-function +describe('GitRepositoryInspectionAdapter', () => { + it('streams object metadata across arbitrary output chunk boundaries', async () => { + const plumbing = mockPlumbing({ + executeStream: vi + .fn() + .mockResolvedValue( + gitStream([ + Buffer.from(`${'a'.repeat(40)} blob 12 9\n${'b'.repeat(20)}`), + Buffer.from(`${'b'.repeat(20)} tree 34 21\n`), + ]) + ), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateObjects())).resolves.toEqual([ + { oid: 'a'.repeat(40), type: 'blob', logicalBytes: 12, physicalBytes: 9 }, + { oid: 'b'.repeat(40), type: 'tree', logicalBytes: 34, physicalBytes: 21 }, + ]); + expect(plumbing.executeStream).toHaveBeenCalledWith({ + args: [ + 'cat-file', + '--batch-all-objects', + '--batch-check=%(objectname) %(objecttype) %(objectsize) %(objectsize:disk)', + ], + }); + }); + + it('includes every ref and reflog root in the reachable inventory', async () => { + const plumbing = mockPlumbing({ + executeStream: vi + .fn() + .mockResolvedValue(gitStream([`${'a'.repeat(40)}\n${'b'.repeat(40)}\n`])), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateReachableObjectIds())).resolves.toEqual([ + 'a'.repeat(40), + 'b'.repeat(40), + ]); + expect(plumbing.executeStream).toHaveBeenCalledWith({ + args: ['rev-list', '--all', '--reflog', '--objects', '--no-object-names'], + }); + }); + + it('uses plumbing safe-prune inspection and never constructs a mutating prune command', async () => { + const expiresBefore = '2026-07-01T00:00:00.000Z'; + const plumbing = mockPlumbing({ + inspectPrunableObjects: vi.fn().mockResolvedValue(gitStream([`${'c'.repeat(40)} blob\n`])), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iteratePrunableObjects({ expiresBefore }))).resolves.toEqual([ + { oid: 'c'.repeat(40), type: 'blob' }, + ]); + expect(plumbing.inspectPrunableObjects).toHaveBeenCalledWith({ expiresBefore }); + }); + + it('streams deterministic ref records and obtains reachable disk usage separately', async () => { + const plumbing = mockPlumbing({ + executeStream: vi + .fn() + .mockResolvedValue( + gitStream([`refs/cas/vault\t${'d'.repeat(40)}\nrefs/heads/main\t${'e'.repeat(40)}\n`]) + ), + execute: vi.fn().mockResolvedValue('1234\n'), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateRefs())).resolves.toEqual([ + { ref: 'refs/cas/vault', oid: 'd'.repeat(40) }, + { ref: 'refs/heads/main', oid: 'e'.repeat(40) }, + ]); + await expect(adapter.reachablePhysicalBytes()).resolves.toBe(1234); + expect(plumbing.executeStream).toHaveBeenCalledWith({ + args: ['for-each-ref', '--format=%(refname)%09%(objectname)', 'refs/'], + }); + expect(plumbing.execute).toHaveBeenCalledWith({ + args: ['rev-list', '--all', '--reflog', '--objects', '--disk-usage'], + }); + }); + + it('rejects ref records outside the refs namespace', async () => { + const plumbing = mockPlumbing({ + executeStream: vi + .fn() + .mockResolvedValue(gitStream([`not-a-ref\t${'a'.repeat(40)}\n`])), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateRefs())).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + }); + + it('rejects malformed prunable-object records', async () => { + const plumbing = mockPlumbing({ + inspectPrunableObjects: vi.fn().mockResolvedValue(gitStream(['not-two-fields\n'])), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect( + collect( + adapter.iteratePrunableObjects({ + expiresBefore: '2026-07-01T00:00:00.000Z', + }) + ) + ).rejects.toMatchObject({ code: 'REPOSITORY_INSPECTION_INVALID' }); + }); + + it('rejects malformed Git output instead of guessing repository evidence', async () => { + const plumbing = mockPlumbing({ + executeStream: vi.fn().mockResolvedValue(gitStream(['not-an-object\n'])), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateObjects())).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + }); + }); + + it('rejects a nonzero streamed Git result after consuming its output', async () => { + const plumbing = mockPlumbing({ + executeStream: vi.fn().mockResolvedValue( + gitStream([`${'a'.repeat(40)}\n`], { + code: 128, + stderr: 'fatal: repository inspection failed', + }) + ), + }); + const adapter = new GitRepositoryInspectionAdapter({ plumbing }); + + await expect(collect(adapter.iterateReachableObjectIds())).rejects.toMatchObject({ + code: 'REPOSITORY_INSPECTION_INVALID', + meta: expect.objectContaining({ + operation: 'reachable object inventory', + stderr: 'fatal: repository inspection failed', + }), + }); + }); +}); diff --git a/test/unit/types/declaration-accuracy.test.js b/test/unit/types/declaration-accuracy.test.js index 672eed1f..dd1890d5 100644 --- a/test/unit/types/declaration-accuracy.test.js +++ b/test/unit/types/declaration-accuracy.test.js @@ -39,7 +39,8 @@ describe('Type declaration accuracy', () => { }); it('keeps runtime JSDoc store encryption shapes aligned with declarations', () => { - const encryptionShape = /@param \{\{ scheme\?: 'whole'\|'framed'\|'convergent', frameBytes\?: number, convergent\?: boolean \}\} \[options\.encryption\]/; + const encryptionShape = + /@param \{\{ scheme\?: 'whole'\|'framed'\|'convergent', frameBytes\?: number, convergent\?: boolean \}\} \[options\.encryption\]/; for (const relPath of [ 'src/domain/services/CasService.js', @@ -53,7 +54,7 @@ describe('Type declaration accuracy', () => { const source = read('src/domain/services/ManifestDiff.js'); expect(source).toMatch( - /@typedef\s+\{import\(['"]\.\.\/value-objects\/Manifest\.js['"]\)\.default\}\s+Manifest/, + /@typedef\s+\{import\(['"]\.\.\/value-objects\/Manifest\.js['"]\)\.default\}\s+Manifest/ ); expect(source).toMatch(/@param\s+\{Manifest\}\s+oldManifest/); expect(source).toMatch(/@param\s+\{Manifest\}\s+newManifest/); @@ -96,7 +97,16 @@ describe('Application-storage declaration accuracy', () => { expect(declarations).toContain('export declare class ExpiringSet'); expect(declarations).toContain('export declare class ExpiringMarker'); expect(declarations).toContain('readonly expiringSets: ExpiringSetCapability;'); - expect(declarations).toContain('addIfAbsent(key: string, options: { expiresAt: Date | string })'); + expect(declarations).toContain('readonly diagnostics: DiagnosticsCapability;'); + expect(declarations).toContain('export declare class RepositoryDoctor'); + expect(declarations).toContain('export declare class RepositoryInspectionPort'); + expect(declarations).toContain('export declare class GitRepositoryInspectionAdapter'); + expect(declarations).toContain('readonly detailed?: number;'); + expect(declarations).toContain("prunableInspection: 'dry-run';"); + expect(declarations).toContain('mutatesRepository: false;'); + expect(declarations).toContain( + 'addIfAbsent(key: string, options: { expiresAt: Date | string })' + ); expect(declarations).toContain('iterateMembers(options: { handle: BundleHandleInput })'); expect(declarations).toContain('applicationRefPrefixes?: string[];'); expect(declarations).toContain('parentOids?: string[];');