From 6c314bfec10595180c74cfe26236b840b96a2ec9 Mon Sep 17 00:00:00 2001 From: Michael Yankelev Date: Sat, 27 Jun 2026 02:02:42 +0200 Subject: [PATCH 1/5] docs: start milestone v2.0 Metadata and Sharing Refactor Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01SDNQVLoAw4DbPrPvtQPobh Entire-Checkpoint: a307f55733df --- .planning/PROJECT.md | 64 +++++++++++++++++++++++++++++++++++++++++--- .planning/STATE.md | 26 +++++++++--------- 2 files changed, 73 insertions(+), 17 deletions(-) diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md index e3a1f813e7..16df8e14df 100644 --- a/.planning/PROJECT.md +++ b/.planning/PROJECT.md @@ -8,11 +8,28 @@ CipherBox is a production-grade, privacy-first encrypted cloud storage platform **Zero-knowledge privacy**: Files are encrypted client-side before leaving the device, and encryption keys exist only in client memory. The server is cryptographically unable to access user data. +## Current Milestone: v2.0 Metadata and Sharing Refactor + +**Goal:** Replace the DB-driven `share_keys` sharing model with metadata-driven read key-chaining (`node/v3`), and close the two confirmed revocation gaps — lazy/unsound read-revocation and un-rotatable write delegation. + +**Target features:** + +- Unified `Node` metadata model (folder/file/root) with two independently sealed bodies (read-body + write-body) and content self-sealing — replaces `FolderMetadata`/`FileMetadata`/`FilePointer`/`FolderEntry` and enables single-file shares +- AAD-bound AES-GCM seal primitive (`sealAesGcmAad`/`unsealAesGcmAad` + `buildNodeAad`) with a frozen byte encoding and a TS↔Rust cross-language KAT +- Read key-chaining: one ECIES at the share-root, then `O(depth)` symmetric AES down the tree — no `share_keys` fan-out, `O(recipients)` grant rows only +- Resumable read-rotation engine (`rotateReadFromNode`) backing read-revoke and every scope-exit mutation — crash-safe, idempotent, with CRIT-1 content-key rotation, M1 generation downgrade defense, HIGH-3 multi-rooted grant re-mint, HIGH-4 add-during-rotation merge +- Unified scope-exit rule (rotate iff a node leaves a grantee's reachable scope; no covering grant ⇒ pure relink) across delete/move/rename, including bin re-link and invite claim re-wrap +- Write-revocation via (c) full Ed25519 rotation (ADR 0001) with rotated-out IPNS name tombstoning +- Resolve/republish/TEE contract rewrite: DB-canonical resolve with `generation` + seq-floor as anti-rollback authority, TEE as a record-lease-renewer (no CID origination, no sequence increment), atomic publish CAS, hardened enclave bindings +- Schema/DB cutover: delete `share_keys`, slim `shares` to `readDescriptorRef`/`writeDescriptorRef`, rename `folder_ipns` → `ipns_records`, drop `folder_ipns.public_key` + +**Source of truth:** `.planning/design/2026-06-26-sharing-read-keychaining-design.md`, [`docs/adr/0001`](../docs/adr/0001-write-revocation-full-ed25519-rotation.md), [`docs/adr/0002`](../docs/adr/0002-read-revocation-protects-future-content-only.md), and the [`CONTEXT.md`](../CONTEXT.md) glossary. Greenfield — no production data, staging wiped — so `node/v3` is the sole codec (no dual-codec bridge, terminology renamed cleanly). + ## Current State **v1.1 IPFS Infrastructure — SHIPPED 2026-06-27** (Milestone 3; 45 phases, 198 plans). All 77 requirements (66 formal v1.1 + 11 HARD) code-satisfied; integration 12/12 WIRED, E2E flows 4/4 INTACT. Delivered self-hosted Someguy IPNS routing (replacing delegated-ipfs.dev), vault blob v2 with DB crypto columns dropped, BYO-IPFS server-relay support, performance baselines + instrumentation, TS+Rust SDK extraction, writable shares, and a cross-layer fail-closed IPNS signature-verify hardening block. See `.planning/milestones/v1.1-MILESTONE-AUDIT.md` for the close-out verdict. -**Next up:** Milestone 4 — Encrypted Productivity Suite (v2.0). Not yet scoped. Fresh requirements come from `/gsd-new-milestone`. +**Next up:** Milestone 4 — v2.0 Metadata and Sharing Refactor (in planning). Scope locked to Tier 1 (read chain + rotation) + Tier 2 (write-revocation + TEE/resolve contract) of the read key-chaining design; Tier 3 capability layer and the Encrypted Productivity Suite are deferred. Requirements derived from the design in `/gsd-new-milestone`. ## Requirements @@ -60,13 +77,27 @@ Full requirement IDs archived in `.planning/milestones/v1.1-REQUIREMENTS.md` (77 ### Active -- [ ] Phase 39 D-02 — add a confirmation dialog before permanent/hard delete (data-safety UX gap; captured todo) +v2.0 Metadata and Sharing Refactor — full requirement list with REQ-IDs in `.planning/REQUIREMENTS.md` (NODE / CRYPTO / READ / ROT / WRITE / TEE / DATA categories). Scope: Tier 1 + Tier 2 of the read key-chaining design. + +Carried from v1.1 (deferred, not yet retired): + +- [ ] Phase 39 D-02 — add a confirmation dialog before permanent/hard delete (data-safety UX gap; captured todo). Note: the bin re-link rework under `node/v3` (DATA-/ROT-) touches this surface. - [ ] Phase 39 D-06 — remove or document the residual server-side `RECYCLE_BIN_RETENTION_DAYS` endpoint/env var (dead backend surface) - [ ] HARD-11 — complete the Phase 60 staging operational smoke-test (D-12 lockstep checkpoint), or accept as an infra-limited override +### Out of Scope (Milestone 4 — v2.0) + +- Tier 3 capability layer (read-side TTL, op-count caps, `capabilityId`) — read-side TTL/op-caps are cryptographically unenforceable; do not add `ttl`/`opCap`/`capabilityId` to `Node` or `SealedChildRef` +- Data migration / dual-codec bridge — greenfield (no prod data, staging wiped); `node/v3` is the sole codec +- Mediated write signing — approach (a)/(d) `POST /ipns/sign` endpoint — runner-up only; (c) full Ed25519 rotation is ratified (ADR 0001) +- Retroactive content protection — read-revoke protects future content/navigation only; already-distributed CIDs and prior versions stay readable (ADR 0002) +- Lazy rotation *walk* (rotate-on-next-write across a subtree) — eager walk is the committed model; the `rotateOne` primitive stays amortizable later +- SEED-001 Phala TEE on-demand cost cycling — separable infra-cost optimization; deferred to a future infra milestone (stays dormant in `.planning/seeds/`) +- Encrypted Productivity Suite (billing, teams, doc editors, signing) — deferred to a later milestone (was tentatively v2.0; v2.0 is now the sharing refactor) + ### Out of Scope (Milestone 3 — v1.1) -- Encrypted Productivity Suite (billing, teams, doc editors, signing) — deferred to Milestone 4 (v2.0) +- Encrypted Productivity Suite (billing, teams, doc editors, signing) — deferred to a post-v2.0 milestone - Mobile apps (iOS/Android) — deferred to Milestone 4+ - Real-time collaborative editing — deferred to Milestone 4+ - Offline write queue / selective sync — deferred to Milestone 4+ @@ -131,7 +162,32 @@ Full requirement IDs archived in `.planning/milestones/v1.1-REQUIREMENTS.md` (77 | Vault blob v2 — zero DB crypto | rootFolderKey in IPFS blob, all DB crypto columns dropped | ✓ | | Monorepo SDK extraction (TS + Rust) | Reusable core shared across web, desktop, recovery tooling | ✓ | | Strict fail-closed IPNS verified-resolver | Single signature-verify chokepoint across all layers | ✓ | +| Metadata-driven read key-chaining (node/v3) | One ECIES at share-root + O(depth) AES; kills share_keys fan-out | — Pending | +| Two sealed bodies per node (read + write) | Structural read/write separation; read grant never conveys signing key | — Pending | +| Write-revocation = (c) full Ed25519 rotation | No new TEE/relay trust; key-possession auth (ADR 0001) | — Pending | +| Read-revoke protects future content only | IPFS is content-addressed; honest threat model (ADR 0002) | — Pending | +| TEE = record-lease-renewer (no CID, no seq++) | Closes republisher stale-CID rollback structurally | — Pending | +| Greenfield node/v3 sole codec | No prod data, staging wiped; no dual-codec/migration bridge | — Pending | + +## Evolution + +This document evolves at phase transitions and milestone boundaries. + +**After each phase transition** (via `/gsd-transition`): + +1. Requirements invalidated? → Move to Out of Scope with reason +2. Requirements validated? → Move to Validated with phase reference +3. New requirements emerged? → Add to Active +4. Decisions to log? → Add to Key Decisions +5. "What This Is" still accurate? → Update if drifted + +**After each milestone** (via `/gsd-complete-milestone`): + +1. Full review of all sections +2. Core Value check — still the right priority? +3. Audit Out of Scope — reasons still valid? +4. Update Context with current state --- -Last updated: 2026-06-27 after v1.1 milestone. v1.1 IPFS Infrastructure (Milestone 3) shipped — 45 phases / 198 plans, 77/77 requirements code-satisfied (66 formal + 11 HARD), integration 12/12, flows 4/4. Remaining carried items: Phase 39 D-02 permanent-delete confirmation (captured todo), D-06 residual server env-var, and the HARD-11 staging operational smoke-test. Next milestone (4 — Encrypted Productivity Suite) not yet scoped; run `/gsd-new-milestone`. See `.planning/milestones/v1.1-MILESTONE-AUDIT.md`. +Last updated: 2026-06-27 — Milestone 4 (v2.0 Metadata and Sharing Refactor) started. Scope locked to Tier 1 (read key-chaining + resumable read-rotation) + Tier 2 (write-revocation via ADR 0001 full Ed25519 rotation + resolve/republish/TEE contract rewrite). Tier 3 capability layer and SEED-001 TEE cost-cycling deferred. Greenfield `node/v3` sole codec. Requirements in `.planning/REQUIREMENTS.md`; design source of truth in `.planning/design/2026-06-26-sharing-read-keychaining-design.md` + ADR 0001/0002 + `CONTEXT.md`. v1.1 carried items (Phase 39 D-02/D-06, HARD-11 staging smoke-test) remain open. diff --git a/.planning/STATE.md b/.planning/STATE.md index 2dd57de675..55be189064 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1,16 +1,16 @@ --- gsd_state_version: 1.0 -milestone: v1.1 -milestone_name: milestone -status: Awaiting next milestone -last_updated: "2026-06-26T23:03:12.795Z" -last_activity: 2026-06-26 — Milestone v1.1 completed and archived +milestone: v2.0 +milestone_name: Metadata and Sharing Refactor +status: planning +last_updated: "2026-06-27T00:01:29.438Z" +last_activity: 2026-06-27 progress: - total_phases: 45 - completed_phases: 45 - total_plans: 198 - completed_plans: 198 - percent: 100 + total_phases: 0 + completed_phases: 0 + total_plans: 0 + completed_plans: 0 + percent: 0 --- # Project State @@ -24,10 +24,10 @@ See: .planning/PROJECT.md (updated 2026-03-07) ## Current Position -Phase: Milestone v1.1 complete +Phase: Not started (defining requirements) Plan: — -Status: Awaiting next milestone -Last activity: 2026-06-26 — Milestone v1.1 completed and archived +Status: Defining requirements +Last activity: 2026-06-27 — Milestone v2.0 started ## Deferred Items From 57d732c2eba90bd83a18b08501cffc7bc8142262 Mon Sep 17 00:00:00 2001 From: Michael Yankelev Date: Sat, 27 Jun 2026 02:21:44 +0200 Subject: [PATCH 2/5] docs: define milestone v2.0 requirements Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01SDNQVLoAw4DbPrPvtQPobh Entire-Checkpoint: e8164a5861c9 --- .planning/REQUIREMENTS.md | 119 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+) create mode 100644 .planning/REQUIREMENTS.md diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md new file mode 100644 index 0000000000..3e56fd6e6c --- /dev/null +++ b/.planning/REQUIREMENTS.md @@ -0,0 +1,119 @@ +# Requirements: CipherBox v2.0 Metadata and Sharing Refactor + +**Defined:** 2026-06-27 +**Core Value:** Zero-knowledge privacy — files encrypted client-side; the server is cryptographically unable to access user data. +**Source of truth:** `.planning/design/2026-06-26-sharing-read-keychaining-design.md` + ADR 0001 (write-revocation = full Ed25519 rotation) + ADR 0002 (read-revoke protects future content only) + `CONTEXT.md` glossary. +**Scope:** Tier 1 (read chain + resumable rotation) + Tier 2 (write-revocation + resolve/republish/TEE contract). Tier 3 out. Greenfield — `node/v3` is the sole codec, no migration. + +## v1 Requirements (this milestone) + +Requirements for v2.0. Each maps to exactly one roadmap phase. Categories: CRYPTO, NODE, READ, ROT, WRITE, TEE, DATA, TEST. + +### CRYPTO — AAD-bound seal primitive + +- [ ] **CRYPTO-01**: `packages/crypto` exposes `sealAesGcmAad`/`unsealAesGcmAad` + a canonical `buildNodeAad(domain‖nodeId‖kind‖generation‖role)` builder, each seal minting a fresh random IV +- [ ] **CRYPTO-02**: A byte-identical Rust twin lives in `cipherbox_crypto`, with a committed cross-language KAT (frozen byte encoding; `kind` 0x01/0x02/0x03, raw 16-byte uuid, 4-byte BE generation, role ∈ {0x01 body, 0x02 child-readkey, 0x03 content, 0x04 child-writekey}) asserted by both TS and Rust +- [ ] **CRYPTO-03**: A sealed blob replayed under a different `childId`/`role`/`generation` fails to unseal (AAD transplant resistance) + +### NODE — unified metadata model and codecs + +- [ ] **NODE-01**: A single `Node` model (folder/file/root via `kind`) with two independently sealed bodies — read-body under `readKey`, write-body under a separate `writeKey` — replaces `FolderMetadata`/`FileMetadata`/`FilePointer`/`FolderEntry` +- [ ] **NODE-02**: A file node's `content` (incl. `content.fileKey`, and each `VersionEntry`'s inline `fileKey` + mandatory `encryptionMode` GCM/CTR) self-seals under the file node's own `readKey` +- [ ] **NODE-03**: `SealedChildRef` is the read-only chain link (`name`, `ipnsName`, `generation` mirror, `versionFloor`, `readKeySealed`); the write link lives in the parent write-body, never in `SealedChildRef` +- [ ] **NODE-04**: The published object is a plaintext envelope (`kind`/`id`/`generation`/`aeadVersion` + `readSealed`/`writeSealed`) with `generation` folded into AAD and tamper-evident +- [ ] **NODE-05**: In Rust crates, `Node` is a real enum (`Folder { children } / File { content } / Root { children }`), not an `Option`-bag — impossible states unrepresentable +- [ ] **NODE-06**: The vault recovery blob carries two keys — `ECIES(rootReadKey)` + `ECIES(rootWriteKey)` — re-designed (not migrated) for the root node's read + write bodies + +### READ — read key-chaining navigation and sharing + +- [ ] **READ-01**: A user can issue a read grant with one ECIES wrap of the share-root `readKey` + one `shares` row (0 node touches, 0 republishes); granting a single file is identical to granting a deep folder +- [ ] **READ-02**: A grantee can navigate to a depth-`d` child via one ECIES unwrap then `O(depth)` symmetric AES, recovering content key/CID/mode at a file node; the read path distinguishes "soft behind, retry" from "hard revoked" +- [ ] **READ-03**: Adding an item seals the child `readKey` under the parent `readKey` with no per-recipient fan-out; `reWrapForRecipients`/`addShareKeys` are deleted +- [ ] **READ-04**: A move within a grantee's scope is link rewrites only (no re-encrypt), computing exact per-grant scope so benign within-scope moves do not over-rotate +- [ ] **READ-05**: An invite wraps the single share-root `readKey` to an ephemeral key (private half in the URL fragment); claim re-wraps it to the claimer's key and stores a standard grant; the `encryptedChildKeys[]` fan-out is deleted + +### ROT — resumable read-rotation and revocation soundness + +- [ ] **ROT-01**: `rotateReadFromNode` is a resumable, per-node-commit, idempotent walk backing read-revoke and every scope-exit mutation; published IPNS records are the source of truth (job record advisory) +- [ ] **ROT-02**: Rotation fires iff a node leaves a grantee's reachable scope; a node with no covering grant is a pure relink (zero rotations) — enforced as a hard test across delete/move/rename +- [ ] **ROT-03**: (CRIT-1) Rotating a file node mints a new `fileKey` (lazy `contentRekeyPending`); a holder of the old `readKey`/`fileKey` cannot decrypt the next published version +- [ ] **ROT-04**: (HIGH-3) Rotation re-mints `readDescriptorRef` for every non-revoked grant whose `rootNodeId` is in the rotated set — no orphaned inner grant +- [ ] **ROT-05**: (HIGH-4) On a CAS-409 the walk re-fetches and re-merges `SealedChildRef`s rather than re-sealing from a stale child list — a concurrent add is never silently dropped +- [ ] **ROT-06**: A crash mid-walk is recoverable — `verifySubtreeClean` rebuilds the frontier, re-run converges, no incorrect double-bump, and the revoked recipient is cut from the root after the root step +- [ ] **ROT-07**: (M1) A durable client-side `{nodeId → highestGeneration}` high-water (survives restart, seeded from the grant `rootGeneration`) fails closed on generation regression + +### WRITE — write-revocation (Tier 2, ADR 0001) + +- [ ] **WRITE-01**: The write-body holds the node's Ed25519 signing material sealed under a separate `writeKey` as a structured recursive write chain (parent seals child `writeKey`, role `0x04`); a read-only holder can never reach signing material +- [ ] **WRITE-02**: Write-revocation performs (c) full Ed25519 rotation — new keypair + k51 name per node, cascading parent re-points to the share root, re-pointing co-grants and owner devices +- [ ] **WRITE-03**: Surviving co-writers receive the rotated Ed25519 key re-wrapped into their `writeDescriptorRef`; an offline co-writer cannot write until re-fetch (explicit) +- [ ] **WRITE-04**: A rotated-out IPNS name is tombstoned (row kept) — the publish gate rejects all writes to it including the EOL-only renewal, resolve returns a tombstone/410, and the name is removed from the TEE republish batch + +### TEE — resolve, republish, and the TEE signing contract (Tier 2) + +- [ ] **TEE-01**: The TEE is a record-lease-renewer — it receives the marshaled `signedRecord`, verifies its signature, and re-emits the same CID and same sequence with only a later EOL; it cannot originate or repoint a CID +- [ ] **TEE-02**: Republish never increments the sequence (the `+ 1n` republisher path is unified to no-increment); sequence-increment policy lives in the relay +- [ ] **TEE-03**: The canonical `ipns_records` row is the sole source of the TEE's signing inputs; `ipns_republish_schedule`'s duplicated `latestCid`/`sequenceNumber`/`encryptedIpnsKey`/`keyEpoch` columns are collapsed +- [ ] **TEE-04**: Publish is an atomic compare-and-set (`UPDATE … WHERE ipnsName = :n AND sequenceNumber = :expected`; 0 rows ⇒ 409); the EOL-only renewal is guarded identically so it can never regress `latestCid`/`sequenceNumber` +- [ ] **TEE-05**: Resolve anti-rollback uses `generation` as the authority plus a durable per-node seq high-water and `versionFloor`; DB is canonical with a case-split fail-closed fall-through (expected-null shared-folder rows apply the seq floor; signedRecord-CID ≠ latestCid fails closed) +- [ ] **TEE-06**: Enclave bindings are hardened — internal epoch self-derivation (never the relay's scalars), name↔key binding asserted before emit, and migration durability via a client recovery path +- [ ] **TEE-07**: The publish gate enforces forward-only `generation` per node server-side (defence-in-depth, mirroring the sequence anti-rollback) + +### DATA — schema/DB cutover and bin + +- [ ] **DATA-01**: The `share_keys` table and entity are deleted outright (no dual-codec, no `version`-discriminator bridge) +- [ ] **DATA-02**: `shares` is slimmed to one grant row per recipient carrying `readDescriptorRef`/`writeDescriptorRef` (legacy `readKeyEcies`/`ShareGrant` retired) +- [ ] **DATA-03**: `folder_ipns` is renamed to `ipns_records` (entity `IpnsRecord`) and `folder_ipns.public_key` is dropped — the Ed25519 pubkey is always recovered from the k51 name via `publicKeyFromIpnsName` +- [ ] **DATA-04**: A `BinEntry` is a `readKey`-sealed re-link; restore is a pure re-link (the `originalFolderKeyEncrypted` re-encrypt-on-restore path is deleted), a private delete is unlink + `BinEntry` (no rotation), and a shared delete rotates the departing subtree + revokes the grant rows + +### TEST — cross-cutting verification infrastructure + +- [ ] **TEST-01**: A rotation crash-safety/resume suite (the must-exist-before-merge suite) extends `tests/sdk-e2e` — the only real client→API IPNS publish/resolve round-trip — with abort-and-resume cases +- [ ] **TEST-02**: The TS↔Rust AAD KAT is a single committed fixture asserted by both `packages/crypto/__tests__` and a Rust `#[test]` (a byte mismatch is silent total decryption failure) +- [ ] **TEST-03**: The winfsp read-path is validated via `Cargo Check & Test (Windows)` (authoritative) and the dispatch-gated desktop E2E is triggered explicitly + +## Future Requirements (deferred) + +### Capability layer (Tier 3) + +- **CAP-01**: Write-plane time-boxing / op-count caps (`ttl`/`opCap`/`capabilityId` on the grant row) — only meaningful on the write path, only if a mediated mechanism is ever chosen; read-side TTL is cryptographically unenforceable. Do NOT add to `Node`/`SealedChildRef`. +- **CAP-02**: Per-file "re-encrypt now" + `O(versions)` "purge history" for high-sensitivity content rotation +- **CAP-03**: Lazy rotation *walk* (rotate-on-next-write across a subtree) — the `rotateOne` primitive is amortizable later if the eager cost proves painful + +### Infra + +- **INFRA-01**: SEED-001 Phala TEE on-demand cost cycling (stop/start the CVM around the republish window) + +## Out of Scope + +Explicitly excluded; documented to prevent scope creep. + +| Feature | Reason | +| --- | --- | +| Data migration / dual-codec bridge | Greenfield — no prod data, staging wiped; `node/v3` is the sole codec | +| Mediated write signing (`POST /ipns/sign`, approach a/d) | Runner-up only; (c) full Ed25519 rotation is ratified (ADR 0001); turns the untrusted relay into a signing oracle | +| Read-side TTL / op-caps | Cryptographically unenforceable — once a reader holds key + CID, IPFS serves it forever | +| Retroactive content protection | Read-revoke protects future content/navigation only; already-distributed CIDs + prior versions stay readable (ADR 0002) | +| Lazy rotation walk | Eager walk is the committed model this milestone | +| Network-first resolve repoint | Stays a post-v2.0 v2 move; near-term DB-canonical with generation + seq-floor authority | +| SEED-001 TEE cost cycling | Separable infra-cost optimization; deferred to a future infra milestone | +| Encrypted Productivity Suite | Deferred to a post-v2.0 milestone | + +## Traceability + +Which phases cover which requirements. Populated during roadmap creation. + +| Requirement | Phase | Status | +| --- | --- | --- | +| (populated by roadmapper) | — | Pending | + +**Coverage:** + +- v1 requirements: 36 total +- Mapped to phases: 0 (pending roadmap) +- Unmapped: 36 ⚠️ + +--- + +_Requirements defined: 2026-06-27_ +_Last updated: 2026-06-27 after initial definition_ From 7bc9a306e5c1ccda8274b69f115e5ed5e7a15db6 Mon Sep 17 00:00:00 2001 From: Michael Yankelev Date: Sat, 27 Jun 2026 02:37:08 +0200 Subject: [PATCH 3/5] docs: create milestone v2.0 roadmap (9 phases) Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01SDNQVLoAw4DbPrPvtQPobh Entire-Checkpoint: e1a3fb8e78f2 --- .planning/MILESTONES.md | 32 +++--- .planning/REQUIREMENTS.md | 48 +++++++- .planning/ROADMAP.md | 227 +++++++++++++++++++++++++++++++++++++- .planning/STATE.md | 192 +++++++++++++++----------------- 4 files changed, 376 insertions(+), 123 deletions(-) diff --git a/.planning/MILESTONES.md b/.planning/MILESTONES.md index 9095383794..15908c540a 100644 --- a/.planning/MILESTONES.md +++ b/.planning/MILESTONES.md @@ -187,32 +187,36 @@ ## Active Milestone -### Milestone 3: IPFS Infrastructure v1.1 (in progress) +### Milestone 4: v2.0 Metadata and Sharing Refactor (in progress) -**Goal:** Make CipherBox IPFS-native -- replace delegated-ipfs.dev, migrate server-side state to IPFS/IPNS, add BYO-IPFS node support, and establish performance baselines -**Depends on:** Milestone 2 -**Phases:** 18-22 -**Requirements:** 25 (4 IPNS + 6 VAULT + 7 BYO + 8 PERF) +**Goal:** Replace the DB-driven share_keys sharing model with metadata-driven read key-chaining (node/v3), and close the two confirmed revocation gaps -- lazy/unsound read-revocation and un-rotatable write delegation. +**Depends on:** Milestone 3 (v1.1 — shipped 2026-06-27) +**Phases:** 61-69 +**Requirements:** 39 (3 CRYPTO + 6 NODE + 5 READ + 7 ROT + 4 WRITE + 7 TEE + 4 DATA + 3 TEST) **Phase structure:** -- Phase 18: Performance Instrumentation (PERF-01 to PERF-04) -- Phase 19: IPNS Resolution Improvement (IPNS-01 to IPNS-04) -- Phase 20: Vault Migration (VAULT-01 to VAULT-06) -- Phase 21: BYO-IPFS Node Support (BYO-01 to BYO-07) -- Phase 22: Performance Baselines Completion (PERF-05 to PERF-08) +- Phase 61: AAD-Bound Seal Primitive and Cross-Language KAT (CRYPTO-01..03, TEST-02) +- Phase 62: Unified Node Codec — Core Keystone (NODE-01..06) +- Phase 63: Read-Chain Navigation and Rotation Core (READ-01..05, ROT-01..02) +- Phase 64: Rotation Soundness — Revocation Guarantees (ROT-03..06, TEST-01) +- Phase 65: SDK Write-Chain, Bin Re-link, and Invite Claim (WRITE-01..04) +- Phase 66: API Schema Cutover, Publish Gate, and Tombstone (DATA-01..04, TEE-04, TEE-05, TEE-07) +- Phase 67: TEE Lease-Renewer Contract Rewrite (TEE-01..03, TEE-06) +- Phase 68: Web Integration — Rotation UX and Durable Client State (ROT-07) +- Phase 69: FUSE and WinFsp — Rust Integration and Grant-Root Awareness (TEST-03) --- ## Future Milestones -### Milestone 4: Encrypted Productivity Suite (planned) +### Milestone 5: Encrypted Productivity Suite (planned) **Goal:** Full encrypted productivity suite -- docs/sheets/slides editors, team accounts, billing (Stripe or crypto), secure document signing, AWS Nitro TEE -**Depends on:** Milestone 3 -**Phases:** 23+ +**Depends on:** Milestone 4 +**Phases:** 70+ --- Created: 2026-02-11 -Last updated: 2026-03-07 after v1.1 IPFS Infrastructure roadmap created +Last updated: 2026-06-27 after v2.0 Metadata and Sharing Refactor roadmap created diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 3e56fd6e6c..faac1b5a7c 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -105,15 +105,53 @@ Which phases cover which requirements. Populated during roadmap creation. | Requirement | Phase | Status | | --- | --- | --- | -| (populated by roadmapper) | — | Pending | +| CRYPTO-01 | Phase 61 | Pending | +| CRYPTO-02 | Phase 61 | Pending | +| CRYPTO-03 | Phase 61 | Pending | +| TEST-02 | Phase 61 | Pending | +| NODE-01 | Phase 62 | Pending | +| NODE-02 | Phase 62 | Pending | +| NODE-03 | Phase 62 | Pending | +| NODE-04 | Phase 62 | Pending | +| NODE-05 | Phase 62 | Pending | +| NODE-06 | Phase 62 | Pending | +| READ-01 | Phase 63 | Pending | +| READ-02 | Phase 63 | Pending | +| READ-03 | Phase 63 | Pending | +| READ-04 | Phase 63 | Pending | +| READ-05 | Phase 63 | Pending | +| ROT-01 | Phase 63 | Pending | +| ROT-02 | Phase 63 | Pending | +| ROT-03 | Phase 64 | Pending | +| ROT-04 | Phase 64 | Pending | +| ROT-05 | Phase 64 | Pending | +| ROT-06 | Phase 64 | Pending | +| TEST-01 | Phase 64 | Pending | +| WRITE-01 | Phase 65 | Pending | +| WRITE-02 | Phase 65 | Pending | +| WRITE-03 | Phase 65 | Pending | +| WRITE-04 | Phase 65 | Pending | +| DATA-01 | Phase 66 | Pending | +| DATA-02 | Phase 66 | Pending | +| DATA-03 | Phase 66 | Pending | +| DATA-04 | Phase 66 | Pending | +| TEE-04 | Phase 66 | Pending | +| TEE-05 | Phase 66 | Pending | +| TEE-07 | Phase 66 | Pending | +| TEE-01 | Phase 67 | Pending | +| TEE-02 | Phase 67 | Pending | +| TEE-03 | Phase 67 | Pending | +| TEE-06 | Phase 67 | Pending | +| ROT-07 | Phase 68 | Pending | +| TEST-03 | Phase 69 | Pending | **Coverage:** -- v1 requirements: 36 total -- Mapped to phases: 0 (pending roadmap) -- Unmapped: 36 ⚠️ +- v1 requirements: 39 total (CRYPTO ×3, NODE ×6, READ ×5, ROT ×7, WRITE ×4, TEE ×7, DATA ×4, TEST ×3) +- Mapped to phases: 39 +- Unmapped: 0 ✓ --- _Requirements defined: 2026-06-27_ -_Last updated: 2026-06-27 after initial definition_ +_Last updated: 2026-06-27 — traceability table populated, coverage 39/39_ diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index be55c50c9f..59e03c04e3 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1,9 +1,9 @@ -# Roadmap: CipherBox v1.1 IPFS Infrastructure +# Roadmap: CipherBox ## Milestones - ✅ **v1.1 IPFS Infrastructure** — Phases 18–60 (shipped 2026-06-27) — full detail: `milestones/v1.1-ROADMAP.md` -- 📋 **Milestone 4: Encrypted Productivity Suite** — planned (see MILESTONES.md) +- 📋 **v2.0 Metadata and Sharing Refactor** — Phases 61–69 (active) ## Phases @@ -58,6 +58,227 @@ +### v2.0 Metadata and Sharing Refactor (Phases 61–69) + +- [ ] **Phase 61: AAD-Bound Seal Primitive and Cross-Language KAT** — Additive AES-GCM+AAD seal in `packages/crypto` and `crates/crypto` with a committed TS↔Rust known-answer test +- [ ] **Phase 62: Unified Node Codec (Core Keystone)** — `Node`/`SealedChildRef`/`PublishedNode` types replacing all legacy metadata types; nothing downstream typechecks until this lands +- [ ] **Phase 63: Read-Chain Navigation and Rotation Core** — Read key-chain walk, `rotateReadFromNode`/`rotateOne` engine, scope-exit predicate, and invite re-wrap in `packages/sdk-core` +- [ ] **Phase 64: Rotation Soundness — Revocation Guarantees** — CRIT-1 content-key rotation, HIGH-3 inner grant re-mint, HIGH-4 concurrent-add merge, crash-safe resume, and the `tests/sdk-e2e` crash-safety suite +- [ ] **Phase 65: SDK Write-Chain, Bin Re-link, and Invite Claim** — Structured write-body, (c) full Ed25519 write-revocation, bin restore as pure re-link, invite claim re-wrap; delete `addShareKeys`/`reWrapForRecipients`/`encryptedChildKeys` +- [ ] **Phase 66: API Schema Cutover, Publish Gate, and Tombstone** — Delete `share_keys`, slim `shares`, rename `folder_ipns` → `ipns_records`, drop `public_key`, atomic CAS publish, tombstone state, resolve case-split, server-side generation gate; run `pnpm api:generate` +- [ ] **Phase 67: TEE Lease-Renewer Contract Rewrite** — TEE becomes a record-lease-renewer (no CID origination, no sequence increment), internal epoch derivation, name↔key binding, tombstone guard +- [ ] **Phase 68: Web Integration — Rotation UX and Durable Client State** — Replace `executeLazyRotation` with `rotateReadFromNode`, durable IndexedDB generation + seq high-water (M1 defense, survives restart), `folderTree` reconcile-before-rotate +- [ ] **Phase 69: FUSE and WinFsp — Rust Integration and Grant-Root Awareness** — Symmetric child-key unwrap, `spawn_file_meta_reencrypt` deletion from both callers, grant-root scope computation, durable client floors, `Node` Rust enum, Windows CI gate + +## Phase Details + +### Phase 61: AAD-Bound Seal Primitive and Cross-Language KAT + +**Goal**: The canonical AES-GCM+AAD seal primitive and its frozen byte encoding exist in both TypeScript and Rust with a committed known-answer test proving byte-identical output. + +**Depends on**: Phase 60 (v1.1 complete) + +**Requirements**: CRYPTO-01, CRYPTO-02, CRYPTO-03, TEST-02 + +**Success Criteria** (what must be TRUE): + +1. `sealAesGcmAad`/`unsealAesGcmAad`/`buildNodeAad` are exported from `packages/crypto` with each seal minting a fresh random IV +2. A byte-identical Rust twin exists in `crates/crypto` with the same AAD encoding (domain separator, raw UUID bytes, 4-byte BE generation, role bytes 0x01–0x04) +3. The cross-language KAT fixture — a single hardcoded vector covering all four role bytes — is asserted by both `packages/crypto/__tests__/build-node-aad.test.ts` AND a Rust `#[test]` in `crates/crypto/tests/cross_language.rs`; both pass in CI +4. A sealed blob replayed under a different `childId`, `role`, or `generation` fails to unseal (AAD transplant resistance test passes) + +**Plans**: TBD + +--- + +### Phase 62: Unified Node Codec (Core Keystone) + +**Goal**: The unified `Node`/`SealedChildRef`/`PublishedNode` types and codecs exist in `packages/core`, replacing all `FolderMetadata`/`FileMetadata`/`FilePointer`/`FolderEntry` types; all downstream packages typecheck after `dist/` rebuild. + +**Depends on**: Phase 61 + +**Requirements**: NODE-01, NODE-02, NODE-03, NODE-04, NODE-05, NODE-06 + +**Success Criteria** (what must be TRUE): + +1. A single `Node` discriminated by `kind` (folder/file/root) carries two independently sealed bodies — `readSealed` under `readKey` and `writeSealed` under `writeKey` — and the published envelope exposes `generation` plaintext as the AAD epoch and anti-rollback witness +2. A file node's `content` (including `content.fileKey` and each `VersionEntry`'s inline `fileKey` + mandatory `encryptionMode`) self-seals under the file node's own `readKey`, not the parent's key +3. `SealedChildRef` contains name, `ipnsName`, `generation` mirror, `versionFloor`, and `readKeySealed` only — the write link is in the parent write-body exclusively +4. Vault recovery blob carries `ECIES(rootReadKey)` + `ECIES(rootWriteKey)` (two keys, one blob); old `encryptedRootFolderKey` field is removed +5. `packages/sdk-core`, `packages/sdk`, and `apps/web` typecheck cleanly after `packages/core` `dist/` is rebuilt — zero references to retired `FolderMetadata`/`FileMetadata`/`FilePointer`/`FolderEntry` +6. `METADATA_SCHEMAS.md` is updated to document the `generation`-as-convergence-witness invariant and the `fileKey`-inside-sealed-read-body semantic change + +**Plans**: TBD + +--- + +### Phase 63: Read-Chain Navigation and Rotation Core + +**Goal**: The read key-chain navigation and rotation walk exist in `packages/sdk-core` as named implementation files; read grants require one ECIES unwrap then O(depth) symmetric AES; the scope-exit predicate gates every delete/move/rename. + +**Depends on**: Phase 62 + +**Requirements**: READ-01, READ-02, READ-03, READ-04, READ-05, ROT-01, ROT-02 + +**Open question (Q2)**: Document whether a large eager rotation in a browser-only (no desktop) session is acceptable — the rotation host question for pure-web users. Decision captured in the phase context file. + +**Success Criteria** (what must be TRUE): + +1. A read grant is issued by ECIES-wrapping the share-root `readKey` into one `shares` row (`readDescriptorRef`) with zero node touches and zero republishes; granting a single file is structurally identical to granting a deep folder +2. A grantee navigates to a depth-`d` child via one ECIES unwrap then `d` symmetric `unsealAesGcmAad` calls, recovering the content key and CID at a file node; the read path distinguishes "soft behind, retry" from "hard revoked" without ambiguity +3. Adding an item seals the child `readKey` under the parent `readKey` with no per-recipient fan-out; `reWrapForRecipients` and `addShareKeys` are deleted from the codebase +4. A move within a grantee's scope produces link rewrites only (zero re-encryption); the scope-exit predicate `hasCoveringGrant` is present and gates every delete/move/rename — a private delete with no active grants triggers zero `rotateReadFromNode` invocations and zero IPNS publishes beyond the parent relink (test verifies zero publish calls) +5. `rotateReadFromNode` is implemented in a named file (`src/rotation/engine.ts` or equivalent, not `index.ts` barrel) so vitest coverage counts it; `rotateOne` commits per-node atomically via CAS before advancing the walk frontier + +**Plans**: TBD + +--- + +### Phase 64: Rotation Soundness — Revocation Guarantees + +**Goal**: Rotation correctly closes all three cryptographic revocation gaps — content-key rotation (CRIT-1), inner-grant re-mint (HIGH-3), concurrent-add merge (HIGH-4) — and survives a crash mid-walk; the `tests/sdk-e2e` crash-safety suite gates the phase. + +**Depends on**: Phase 63 + +**Requirements**: ROT-03, ROT-04, ROT-05, ROT-06, TEST-01 + +**Success Criteria** (what must be TRUE): + +1. (CRIT-1 / §7.3 test 2) Rotating a file node mints a new `fileKey'` and sets `contentRekeyPending`; a test asserts a holder of the old `readKey`/`fileKey` cannot decrypt the next published version of the file +2. (HIGH-3 / §7.3 test 3) Rotation queries `shares WHERE rootNodeId IN (rotated_node_ids)` and re-mints `readDescriptorRef` for every non-revoked recipient including inner grants rooted at subtree nodes; a test with a leaf-level share asserts the inner grantee's descriptor is re-minted and the revoked recipient's row is deleted +3. (HIGH-4 / §7.3 test 4) On a CAS-409, `rotateOne` re-fetches the current parent node, re-decodes the read-body, and merges concurrently-added `SealedChildRef`s before re-sealing; a test injects a concurrent upload mid-rotation and asserts the new child is present in the completed parent +4. A crash mid-walk is recovered by re-running `rotateReadFromNode`; `verifySubtreeClean` rebuilds the frontier from published IPNS records, re-run converges without double-bumping any node's `generation`, and the revoked recipient is cut from the root after the root step +5. (TEST-01) The `tests/sdk-e2e` abort-and-resume suite covering crash-safety passes against a live local API stack; SDK E2E must pass before phase sign-off (it is the only real client→API IPNS publish/resolve round-trip) + +**Plans**: TBD + +--- + +### Phase 65: SDK Write-Chain, Bin Re-link, and Invite Claim + +**Goal**: The write-body carries Ed25519 signing material sealed under a separate `writeKey`; write-revocation performs full Ed25519 rotation per ADR 0001; bin restore is a pure re-link; invite claim re-wraps a single root `readKey`. + +**Depends on**: Phase 64 + +**Requirements**: WRITE-01, WRITE-02, WRITE-03, WRITE-04 + +**Open question (Q3)**: When a write-recipient deletes or moves a node the owner independently sub-shared, the unlink and the revocation split across two principals. Decide the authority model and the acceptable exposure window; document the decision in the phase context file. + +**Success Criteria** (what must be TRUE): + +1. The write-body holds the node's Ed25519 signing material sealed under `writeKey` with role `0x04` (`child-writekey`); a read-only holder who holds only `readDescriptorRef` can never reach signing material — verified by attempting to unseal the write-body with only the `readKey` +2. Write-revocation generates a new Ed25519 keypair and k51 name per node, cascading parent re-points to the share root; old names are tombstoned (publish gate rejects, resolve returns 410) and removed from the TEE republish batch +3. Surviving co-writers receive the rotated Ed25519 key re-wrapped into their `writeDescriptorRef`; an offline co-writer receives a clear "cannot write until re-fetch" error on next attempt +4. `bin` restore is a pure re-link (`BinEntry` re-sealed under destination `readKey`); `originalFolderKeyEncrypted` and its re-encrypt-on-restore path are deleted from `packages/core/src/bin/types.ts` and `packages/sdk/src/bin/index.ts`; `encryptedChildKeys` JSONB fan-out is deleted from invite claim + +**Plans**: TBD + +--- + +### Phase 66: API Schema Cutover, Publish Gate, and Tombstone + +**Goal**: The database reflects the `node/v3` model: `share_keys` deleted, `shares` slimmed to descriptor refs, `folder_ipns` renamed to `ipns_records` with `public_key` dropped, atomic CAS publish, tombstone state machine, and case-split resolve hardening. + +**Depends on**: Phase 65 + +**Requirements**: DATA-01, DATA-02, DATA-03, DATA-04, TEE-04, TEE-05, TEE-07 + +**Sub-phase research flag**: Before writing the TypeORM migration, inspect the live FK constraint map for `folder_ipns` → `ipns_records` rename against staging DB schema; all referencing tables (`ipns_republish_schedule`, `shares`, `vaults`) must migrate atomically. + +**Success Criteria** (what must be TRUE): + +1. `share_keys` table and entity are deleted; `shares` carries `readDescriptorRef`/`writeDescriptorRef`/`rootNodeId`/`rootIpnsName`/`rootGeneration`; the legacy `readKeyEcies`/`ShareGrant` shape is gone from all entity, DTO, and service files +2. `folder_ipns` is renamed to `ipns_records` (entity class `IpnsRecord`); the `public_key` column is dropped; strict-verify recovers the Ed25519 pubkey exclusively via `publicKeyFromIpnsName`; a test with a null-`public_key` shared-folder row asserts strict-verify works correctly +3. Publish is an atomic conditional UPDATE (`WHERE ipnsName = :n AND sequenceNumber = :expected`; zero rows ⇒ 409); (§7.3 test 16) two concurrent publishes at the same `dbSeq` produce exactly one 409 and zero lost updates +4. (§7.3 test 15) The `parseCachedRecord`-null case-split is explicit: a legitimate null-`signedRecord` shared-folder row applies the `seq ≥ storedSeq` floor; a `signedRecord`-CID mismatch fails closed — neither falls through ungated +5. A tombstoned `ipns_records` row is rejected at the publish gate (403/410) and at the EOL-only renewal; resolve returns a 410 marker for tombstoned names; server-side `generation` gate enforces forward-only per node, mirroring the sequence CAS +6. `pnpm api:generate` is run and the regenerated `packages/api-client/src/generated/` is committed alongside the migration; the `check-api-client.sh` pre-commit hook passes + +**Plans**: TBD + +--- + +### Phase 67: TEE Lease-Renewer Contract Rewrite + +**Goal**: The TEE worker is a record-lease-renewer — it receives a marshaled `signedRecord`, verifies its signature, and re-emits the same CID and sequence with only a later EOL; it cannot originate or repoint a CID. + +**Depends on**: Phase 66 + +**Requirements**: TEE-01, TEE-02, TEE-03, TEE-06 + +**Success Criteria** (what must be TRUE): + +1. (§7.3 test 12) The `+ 1n` sequence increment is removed from `apps/tee-worker/src/routes/republish.ts`; republish re-signs with the same `sequenceNumber` and same `value` (CID), only a later EOL; a test asserts the re-signed record has equal `sequenceNumber` to the input and that a revoked CID is never re-signed forward +2. The TEE derives `currentEpoch` from its own clock (never from relay-supplied scalars); it asserts `publicKeyFromIpnsName(ipnsName) == pubkey(decryptedKey) == record.pubkey` before emitting any re-signed record; a tombstoned name presented to the renewer is rejected at the publish gate +3. The canonical `ipns_records` row is the sole source of the TEE's signing inputs; `ipns_republish_schedule`'s duplicated `latestCid`/`sequenceNumber`/`encryptedIpnsKey`/`keyEpoch` columns are collapsed; no signing inputs are sourced from the schedule snapshot +4. The EOL-only renewal uses the same atomic CAS guard (`WHERE sequenceNumber = :loaded`), so it can never regress `latestCid`/`sequenceNumber`; a TEE republish E2E round-trip (staging or local stack) confirms the new contract end-to-end + +**Plans**: TBD + +--- + +### Phase 68: Web Integration — Rotation UX and Durable Client State + +**Goal**: The web app uses `rotateReadFromNode` for all revocation-triggering mutations, persists a durable IndexedDB generation + seq high-water that survives page reload, and reconciles `folderTree` before any rotation publish. + +**Depends on**: Phase 67 + +**Requirements**: ROT-07 + +**Open question (Q1)**: A co-writer offline during write-key rotation cannot write until re-fetch. Accept as explicit with a clear error message, or add a grace period/notification? Decision documented in the phase context file. + +**Open question (Q3 — web side)**: When a write-recipient deletes/moves a node the owner independently sub-shared, decide the authority model for the web mutation path (mirrors Phase 65 Q3 decision). + +**Success Criteria** (what must be TRUE): + +1. (M1 / §7.3 test 5) The `{nodeId → highestGeneration}` map persists to IndexedDB; a test simulates a page reload mid-session and asserts generation regression is rejected fail-closed after restart — in-memory-only storage is rejected at review +2. `executeLazyRotation` is deleted from `apps/web/src/services/share.service.ts`; all revocation-triggering paths (delete, move, rename when scope exit) call `rotateReadFromNode`; `addShareKeys` and `reWrapForRecipients` are deleted from per-mutation fan-out paths +3. `folderTree` is reconciled against the current `sequenceNumber` before any rotation publish; if reconciliation fails the mutation defers rather than skipping rotation — the `#489`/`#494` desync class cannot produce a silent missed revocation +4. A durable per-node `{nodeId → highestSeq}` seq high-water is wired into `resolveIpnsRecord` in the web resolve path; a generation or seq regression from the relay causes a fail-closed error, not silent acceptance +5. All new web test files use the `.test.ts` extension (not `.spec.ts`); `find apps/web/src -name "*.spec.ts"` returns empty + +**Plans**: TBD + +**UI hint**: yes + +--- + +### Phase 69: FUSE and WinFsp — Rust Integration and Grant-Root Awareness + +**Goal**: The FUSE and WinFsp clients use symmetric key unwrap throughout, grant-root awareness gates scope-exit mutations, `Node` is a real Rust enum, and the Windows CI gate passes. + +**Depends on**: Phase 68 + +**Requirements**: TEST-03 + +**Sub-phase research flag**: The grant-root scope computation algorithm in `crates/fuse/src/write_ops/` is net-new and under-specified in the design; a plan-time design pass is required before implementation. + +**Open question (Q3 — FUSE side)**: When a write-recipient deletes/moves a node the owner independently sub-shared, decide the authority model for the FUSE delete path (mirrors Phase 65 Q3 decision). + +**Success Criteria** (what must be TRUE): + +1. All `cipherbox_crypto::ecies::unwrap_key` calls in `crates/fuse/src/inode.rs` (lines 434, 452, 658, 716) and `crates/fuse/src/replay.rs` (line 365) are replaced by `cipherbox_crypto::aes::unseal_aes_gcm_aad` symmetric unwrap with correct `buildNodeAad` AAD +2. `spawn_file_meta_reencrypt` is deleted from `crates/fuse/src/metadata.rs` AND from both callers: `crates/fuse/src/write_ops/implementation/rename.rs` (line 248) and `crates/fuse/src/platform/windows/write_ops.rs` (line 1183) — Windows path verified in CI, not locally +3. Grant-root awareness is implemented in `delete`/`rename`/`move` FUSE paths: a shared-scope exit triggers `rotateReadFromNode`; a private delete with no active grants is a pure relink with zero rotation publishes +4. `enum Node { Folder { children: Vec }, File { content: SealedContent }, Root { children: Vec } }` exists in `crates/core/src/`; durable generation + seq high-water is persisted adjacent to the write journal (survives FUSE daemon restart) +5. (TEST-03 / §7.3 test 21) `Cargo Check & Test (Windows)` CI gate passes; the dispatch-gated desktop E2E is triggered explicitly via `gh workflow run "CI E2E Tests" --ref ` and passes before phase sign-off + +**Plans**: TBD + +--- + ## Progress -All 45 phases complete (198 plans). See `milestones/v1.1-ROADMAP.md` for full detail and `milestones/v1.1-MILESTONE-AUDIT.md` for the close-out audit. +| Phase | Name | Plans Complete | Status | Completed | +| --- | --- | --- | --- | --- | +| 61 | AAD-Bound Seal Primitive and Cross-Language KAT | 0/? | Not started | - | +| 62 | Unified Node Codec (Core Keystone) | 0/? | Not started | - | +| 63 | Read-Chain Navigation and Rotation Core | 0/? | Not started | - | +| 64 | Rotation Soundness — Revocation Guarantees | 0/? | Not started | - | +| 65 | SDK Write-Chain, Bin Re-link, and Invite Claim | 0/? | Not started | - | +| 66 | API Schema Cutover, Publish Gate, and Tombstone | 0/? | Not started | - | +| 67 | TEE Lease-Renewer Contract Rewrite | 0/? | Not started | - | +| 68 | Web Integration — Rotation UX and Durable Client State | 0/? | Not started | - | +| 69 | FUSE and WinFsp — Rust Integration and Grant-Root Awareness | 0/? | Not started | - | + +v1.1 history: 45 phases complete (198 plans). See `milestones/v1.1-ROADMAP.md` for full detail. diff --git a/.planning/STATE.md b/.planning/STATE.md index 55be189064..2bfcd1d4e2 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,11 +2,11 @@ gsd_state_version: 1.0 milestone: v2.0 milestone_name: Metadata and Sharing Refactor -status: planning +status: planning_complete last_updated: "2026-06-27T00:01:29.438Z" last_activity: 2026-06-27 progress: - total_phases: 0 + total_phases: 9 completed_phases: 0 total_plans: 0 completed_plans: 0 @@ -17,17 +17,19 @@ progress: ## Project Reference -See: .planning/PROJECT.md (updated 2026-03-07) +See: .planning/PROJECT.md (updated 2026-06-27) **Core value:** Zero-knowledge privacy -- files encrypted client-side, server never sees plaintext -**Current focus:** v1.1 milestone close — finish verification (Phases 59, 60) + re-audit hardening block, then /gsd:complete-milestone v1.1 +**Current focus:** v2.0 Metadata and Sharing Refactor — planning complete, ready to begin Phase 61 ## Current Position -Phase: Not started (defining requirements) +Phase: 61 — AAD-Bound Seal Primitive and Cross-Language KAT Plan: — -Status: Defining requirements -Last activity: 2026-06-27 — Milestone v2.0 started +Status: Planning complete — awaiting `/gsd-plan-phase 61` +Last activity: 2026-06-27 — v2.0 roadmap created (9 phases, Phases 61–69) + +Progress: `░░░░░░░░░░` 0 / 9 phases (0%) ## Deferred Items @@ -46,7 +48,7 @@ Items acknowledged and deferred at v1.1 milestone close on 2026-06-27. None are ## Performance Metrics -**Velocity:** +**Velocity (v1.1):** - Total plans completed: 164 (all 34 milestone v1.1 phases; every PLAN has a SUMMARY) - Average duration: 5.5 min @@ -102,7 +104,7 @@ Items acknowledged and deferred at v1.1 milestone close on 2026-06-27. None are | Phase 25 P01 | 5min | 2 tasks | - | | Phase 25 P02 | 4min | 2 tasks | - | | Phase 25 P03 | - | 2 tasks | - | -| Phase 26 P01 | 5min | 2 tasks | 6 files | +| Phase 26 P01 | 5min | 3 tasks | 6 files | | Phase 26 P02 | 4min | 2 tasks | 5 files | | Phase 27 P01 | 6min | 2 tasks | 23 files | | Phase 27 P02 | 5min | 2 tasks | 4 files | @@ -153,38 +155,38 @@ Items acknowledged and deferred at v1.1 milestone close on 2026-06-27. None are | Phase 41 P05 | 3min | 2 tasks | 3 files | | Phase 45 P01 | 8min | 2 tasks | 2 files | | Phase 45 P02 | 8min | 1 tasks | 3 files | -| Phase 45 P03 | 7min | 2 tasks | 4 files | -| Phase 45 P04 | 8min | 1 tasks | 2 files | -| Phase 45 P05 | 12 | 2 tasks | 1 files | -| Phase 45 P06 | 90 | - tasks | - files | -| Phase 48 P01 | 15min | 3 tasks | 4 files | -| Phase 48 P02 | 2min | 2 tasks | 5 files | -| Phase 48 P03 | 6min | 3 tasks | 7 files | -| Phase 48 P05 | 8min | 3 tasks | 145 files | -| Phase 48 P06 | 18min | 3 tasks | 5 files | -| Phase 49 P01 | 13min | 2 tasks | 5 files | -| Phase 49 P02 | 15min | 1 tasks | 1 files | -| Phase 49 P03 | 11min | 4 tasks | 7 files | -| Phase 49 P04 | 26min | 3 tasks | 5 files | -| Phase 49 P05 | 12min | 2 tasks | 2 files | -| Phase 51 P01 | 9min | 3 tasks | 2 files | -| Phase 51 P03 | 45min | 4 tasks | 12 files | -| Phase 51 P04 | 12min | 3 tasks | 6 files | -| Phase 56 P01 | 45min | 3 tasks | 5 files | -| Phase 56 P02 | 90min | 3 tasks | 8 files | -| Phase 58 P01 | 45 | 5 tasks | 10 files | -| Phase 58-ipns-signature-verify-coverage P02 | 30min | 3 tasks | 2 files | -| Phase 58-ipns-signature-verify-coverage P04 | 25min | 3 tasks | 4 files | -| Phase 59 P01 | 35min | 2 tasks | 2 files | -| Phase 59 P02 | 4min | 2 tasks | 6 files | -| Phase 59 P03 | 12min | 2 tasks | 6 files | -| Phase 59 P04 | 15min | 2 tasks | 7 files | -| Phase Phase 60 P01 P01 | 35min | 2 tasks | 9 files | -| Phase 60 P02 | 4min | 2 tasks | 7 files | -| Phase 60 P03 | 9min | 2 tasks | 3 files | -| Phase 60 P04 | 15min | 2 tasks | 11 files | -| Phase 60 P05 | 16 | - tasks | - files | -| Phase 60 P06 | 14min | 2 tasks | 7 files | +| Phase 45 P03 | 7min | 2 tasks | 4 files | +| Phase 45 P04 | 8min | 1 tasks | 2 files | +| Phase 45 P05 | 12min | 2 tasks | 1 files | +| Phase 45 P06 | 90min | - tasks | - files | +| Phase 48 P01 | 15min | 3 tasks | 4 files | +| Phase 48 P02 | 2min | 2 tasks | 5 files | +| Phase 48 P03 | 6min | 3 tasks | 7 files | +| Phase 48 P05 | 8min | 3 tasks | 145 files | +| Phase 48 P06 | 18min | 3 tasks | 5 files | +| Phase 49 P01 | 13min | 2 tasks | 5 files | +| Phase 49 P02 | 15min | 1 tasks | 1 files | +| Phase 49 P03 | 11min | 4 tasks | 7 files | +| Phase 49 P04 | 26min | 3 tasks | 5 files | +| Phase 49 P05 | 12min | 2 tasks | 2 files | +| Phase 51 P01 | 9min | 3 tasks | 2 files | +| Phase 51 P03 | 45min | 4 tasks | 12 files | +| Phase 51 P04 | 12min | 3 tasks | 6 files | +| Phase 56 P01 | 45min | 3 tasks | 5 files | +| Phase 56 P02 | 90min | 3 tasks | 8 files | +| Phase 58 P01 | 45min | 5 tasks | 10 files | +| Phase 58 P02 | 30min | 3 tasks | 2 files | +| Phase 58 P04 | 25min | 3 tasks | 4 files | +| Phase 59 P01 | 35min | 2 tasks | 2 files | +| Phase 59 P02 | 4min | 2 tasks | 6 files | +| Phase 59 P03 | 12min | 2 tasks | 6 files | +| Phase 59 P04 | 15min | 2 tasks | 7 files | +| Phase 60 P01 | 35min | 2 tasks | 9 files | +| Phase 60 P02 | 4min | 2 tasks | 7 files | +| Phase 60 P03 | 9min | 2 tasks | 3 files | +| Phase 60 P04 | 15min | 2 tasks | 11 files | +| Phase 60 P05 | 16min | - tasks | - files | +| Phase 60 P06 | 14min | 2 tasks | 7 files | ## Accumulated Context @@ -225,7 +227,7 @@ Recent for v1.1: - Recovery tool IPNS resolution uses gateway /ipns/ HEAD request with redirect following (most reliable without API dependency) - fetchAndDecryptMetadata handles both v1 JSON and v2 binary blobs transparently for folder sync - Zero-crypto vault schema: server stores only ownerPublicKey and rootIpnsName, all crypto material lives exclusively in IPFS v2 blobs -- DB crypto columns (encrypted_root_folder_key, encrypted_root_ipns_private_key, migrated_at) fully dropped — no fallback paths +- DB crypto columns (encrypted_root_folder_key, encrypted_root_ipns_private_key, migrated_at) fully dropped -- no fallback paths - PinningProvider interface: KuboProvider uses Basic auth, PsaProvider uses Bearer auth, matching each protocol's native auth model - PsaProvider.pin() throws intentionally; pinByCid() is the correct PSA workflow (CID-reference-only protocol) - Connection test uses sequential probe: Kubo /api/v0/id first, then PSA /pins, with 10s timeout per probe @@ -236,13 +238,13 @@ Recent for v1.1: - PsaProvider.pinByCid() accessed via cast in client.ts (PSA-specific, not on PinningProvider interface) - Migration uses existing BullMQ pattern with pin-migration queue name; TEE decrypts ECIES-encrypted provider configs in-enclave with epoch key - SSRF protection on TEE migration: validates URL structure (HTTPS-only, no private IPs) and DNS resolution (rebinding check) -- BYO config stored as encrypted IPNS entry using rootFolderKey — no server-side credential storage (zero-knowledge preserved) +- BYO config stored as encrypted IPNS entry using rootFolderKey -- no server-side credential storage (zero-knowledge preserved) - Dedicated IPNS key derived via HKDF with context string byo-ipfs-config from vault keypair -- BYO benchmark execution (21-07 Task 4) deferred — requires external IPFS provider infrastructure; test scenarios ready to run when provider available +- BYO benchmark execution (21-07 Task 4) deferred -- requires external IPFS provider infrastructure; test scenarios ready to run when provider available - BYO config loaded at login via IPNS resolve with graceful fallback to cipherbox-only mode - Source unpin is best-effort and non-fatal after verified CID transfer to destination - Cargo workspace with centralized deps at repo root; cipherbox-crypto crate as foundation for all Rust SDK extraction -- Module re-export pattern in desktop crypto/mod.rs preserves all existing crate::crypto::\* paths without touching call sites +- Module re-export pattern in desktop crypto/mod.rs preserves all existing crate::crypto::* paths without touching call sites - cipherbox-core crate layered on cipherbox-crypto: folder, file, bin, vault_blob, ipns, registry, decrypt, error modules - File module re-exports FileMetadata types from folder.rs (shared AES encryption context with parent folder key) - decrypt module moved from fuse to crypto re-export (domain logic, not FUSE-specific) @@ -251,7 +253,7 @@ Recent for v1.1: - Shared test vectors in tests/vectors/ JSON files loadable by both Rust and TypeScript for CI parity gates - SyncDaemon uses Arc generic callback instead of Tauri AppHandle for testability - Desktop api/client.rs re-exports cipherbox_api_client::ApiClient as type alias to unify types across modules -- Desktop AppState wraps Arc from SDK; all key material accessed via state.sdk.\* +- Desktop AppState wraps Arc from SDK; all key material accessed via state.sdk.* - Keychain operations kept as desktop-specific keychain.rs module (not in api-client crate) - Desktop api/ and crypto/ directories fully removed; all imports use workspace crates directly - CI parity gate uses needs.changes.outputs.src (not nonexistent packages) for trigger condition @@ -271,57 +273,62 @@ Recent for v1.1: - Shared file download/view falls back to fileKeyEncrypted from metadata when no share_key exists - FilePointer resolution uses FileMetadata directly (no separate ResolvedFileMetadata struct) - FilePointer resolution scoped to parent folder via get_unresolved_file_pointers_for_parent() to avoid wrong-folder-key decryption -- FilePointer async resolution: 500ms base \* 2^attempt exponential backoff (1s, 2s, 4s) with 3 retries +- FilePointer async resolution: 500ms base * 2^attempt exponential backoff (1s, 2s, 4s) with 3 retries - Removed custom dstack-sdk.d.ts since @phala/dstack-sdk@0.5.7 ships own TypeScript types - Defensive CVM key derivation handles both key (v0.5+) and asUint8Array (legacy) SDK return types - TEE worker Prometheus metrics use `cipherbox_tee_*` prefix for Grafana dashboard coexistence with API metrics - TEE worker structured JSON logger has zero external dependencies (JSON.stringify to stdout/stderr) - [Phase 48-05] Share itemName encrypted at rest via additive nullable item_name_encrypted bytea on BOTH shares and share_invites (decision A3 includes invite flow); migration is additive-only with NO data UPDATE (server zero-knowledge cannot re-encrypt legacy plaintext); itemNameEncrypted optional hex DTO on create-share/create-invite/claim-invite; claim re-wraps ephemeral→recipient ciphertext onto the Share; web encrypt/decrypt/lazy-backfill deferred to 48-06 -- [Phase 48-06] Web ECIES-wraps itemName on share/invite create (recipient pubkey for direct, ephemeral pubkey for invite) and sends ciphertext-only (itemName: '' + itemNameEncrypted) — no plaintext display name at rest for new rows; recipient decrypts itemNameEncrypted into the store's plaintext projection on received-share load so display sites are unchanged; owner sent-list uses plaintext fallback (zero-knowledge: name wrapped for recipient, owner can't decrypt — T-48-18 accept). API GAP: no update endpoint accepts itemNameEncrypted, so the legacy lazy-backfill (A2) is detect+re-wrap only; persist blocked pending a follow-up API plan (PATCH itemNameEncrypted) +- [Phase 48-06] Web ECIES-wraps itemName on share/invite create (recipient pubkey for direct, ephemeral pubkey for invite) and sends ciphertext-only (itemName: '' + itemNameEncrypted) -- no plaintext display name at rest for new rows; recipient decrypts itemNameEncrypted into the store's plaintext projection on received-share load so display sites are unchanged; owner sent-list uses plaintext fallback (zero-knowledge: name wrapped for recipient, owner can't decrypt -- T-48-18 accept). API GAP: no update endpoint accepts itemNameEncrypted, so the legacy lazy-backfill (A2) is detect+re-wrap only; persist blocked pending a follow-up API plan (PATCH itemNameEncrypted) ### Roadmap Evolution - Phase 19.1 inserted after Phase 19: Extract core crypto SDK as shared package (URGENT) -- Phase 19.2 inserted after Phase 19: IPFS Upload Performance Optimization (URGENT) — concurrent pins, Kubo worker tuning, pin batching to address ~95% bottleneck in upload path identified by Phase 19 baselines -- Phase 23 added: Rust SDK Extraction — extract shared cipherbox-core crate, replace duplicated logic in desktop FUSE code, enable unit testing parity with TypeScript -- Phase 27 added: Writable Shares (PoC) — extend read-only sharing to read-write using existing server-coordinated conflict resolution -- Phase 36 added: Refactor upload progress in web app, to an inline progress display and remove the popup upload progress -- Phase 37 added: Parallel batch upload pipeline — replace sequential per-file upload loop with parallel encrypt+pin and single folder metadata update +- Phase 19.2 inserted after Phase 19: IPFS Upload Performance Optimization (URGENT) +- Phase 23 added: Rust SDK Extraction +- Phase 27 added: Writable Shares (PoC) +- Phase 36 added: Inline upload progress +- Phase 37 added: Parallel batch upload pipeline - Phase 41 added: Package and app versioning and release cycles -- Phase 42 added: API unpin integrity — ownership check, cross-user refcount, quota decrement (audit gap closure, todos 2026-06-11) -- Phase 43 added: FUSE write durability — persisted upload journal, mkdir orphan fix (audit gap closure, todos 2026-06-11) -- Phase 44 added: IPNS conflict handling — merge-on-409, file CAS (audit gap closure, todo 2026-06-11) -- Phase 45 added: Desktop FUSE write-durability cleanup — Rust hygiene refactors + test coverage for phase 43/44 journal+replay (todos #11, #12, #14, #15, #18, #19, #20); excludes data-loss bugs #7/#8/#17 -- Phase 46 added 2026-06-15: Desktop FUSE data-loss bugs + replay hardening — the #7/#8/#17 bugs Phase 45 deferred, the two PR #491 replay follow-ups, and the deferred read_ops/write_ops + journal_helpers test coverage (grouped desktop todos) -- Phase 47 added 2026-06-15: SDK folder-state and publish-path consolidation — unify folderTree/Zustand ownership, one publishWithCas CAS-retry, encapsulate baseChildren bookkeeping, fix updateSharedFile prunedCids pin leak (grouped SDK todos) -- Phase 48 added 2026-06-16: SDK self-bootstrap regression fix + shared-folder/metadata consolidation — P0 fix for the PR #498 self-bootstrap clobber regressing main web-e2e (run 27587113911), then remove redundant web folder-seeding (#9, gated on the fix), route shared-folder writes through the SDK client (#8), encrypt share itemName at rest (#5 / Phase-14 M1); defers CRDT-inbox research (#2) -- Phase 49 added 2026-06-18: Shared-folder intra-share move + useFolderNavigation unwrap consolidation — recipient-side move of a file between subfolders within one share (re-encrypts FileMetadata to the dest folderKey via reencryptFileMetadataForFolderChange, mirroring owner moveItem / #507), anywhere-in-subtree destination picker (new SDK shared-subtree enumeration), plus consolidating the duplicate web useFolderNavigation ECIES unwrap onto client.ensureFolderLoaded; closes captured todos #8 + #7; builds on Phase 48 shared-folder ownership -- Milestone v1.1 REOPENED 2026-06-19 with a hardening block (Phases 50–55) absorbing tracked tech-debt/security todos from v1.1 verification and audits: 50 IPFS/IPNS data-integrity (#12 unpin-integrity, #14 unenroll-subtrees); 51 crypto-signature & secret-leak hardening (#5 IPNS sig, #15 web logger redaction/Faro); 52 desktop FUSE durability & at-rest safety (#9); 53 release & supply-chain engineering (#6 pin-actions, #13 cargo-lock, #16 release-please-pins); 54 E2E test-infra typing (#11); 55 large source-file refactor (#17). Reopened into Milestone 3 rather than opening v1.2, since Milestone 4 (v2.0) is already defined. Excludes the GSD-tooling STATE regression (#10 — upstream chore, not product code). Todos #7 (useFolderNavigation consolidation) and #8 (shared-move re-encrypt) were verified already-resolved by Phase 49 (confirmed in live code) and moved to `.planning/todos/completed/`. - -- Phase 56 added 2026-06-21: FUSE & IPNS Durability Hardening (HARD-07) — per-file/bin IPNS Conflict re-resolve/retry, write-path EINVAL/EFBIG/EEXIST guards, key-wrap/decode error propagation, inode identity reset, spawn_metadata_publish zeroization; the pre-existing findings surfaced byte-identical by the PR #538 / Phase 55 refactor review (absorbs the superseded per-file-IPNS-conflict todo) -- Phase 57 added 2026-06-21: API CID/Provider Hardening & Module Dedup (HARD-08) — shared CID_REGEX+MaxLength across RegisterCidDto/UnpinDto, URL-encoded LocalProvider pin/cat URLs, leaf IpfsProviderModule, shared withCidLock/refcountAndMaybeUnpin -- Phase 58 added 2026-06-21: IPNS Signature-Verify Coverage (HARD-09) — CBOR cid/sequence binding + Rust resolve_ipns_verified chokepoint, non-CAS embedded-sequence validation, web/sdk-core resolve dedup, shared verify test vectors; the S1/S2 residue of Phase 51 / PR #529. Master IPNS-sig todo (2026-06-13) and large-file-refactor Tier-1/2 todo (2026-06-19) filed to completed/, the latter with its Tier-3 residue re-captured. -- Phase 59 added 2026-06-23: FUSE IPNS Verify/Publish Hardening & Cleanup (HARD-10) — the Phase 56/58 FUSE long-tail from the 2026-06-23 pending-todo audit: the 2 partial HARD-07 residuals (fs.rs File-branch wrap_key error propagation; inode file-side re-resolution on changed file_meta_ipns_name), VerifyError::Legacy carrying the legacy response, CAS dead journal_entry param + content_ops dead-binding cleanup, phase58 simplify follow-ups, and unifying the first-publish embedded-sequence convention (bridges to 60). Sourced from 6 captured todos. -- Phase 60 added 2026-06-23: IPNS Verification Cross-Layer Closeout — Desktop + API (HARD-11) — route remaining apps/desktop Tauri resolve_ipns sites through the verified resolver (scoped fail-closed), and recover per-op IPNS verify CPU on the API publish/resolve hot path via a safe short-circuit / short-TTL verified-record cache that still fully verifies untrusted/DHT records. Sourced from 2 captured todos: the API verify-caching todo (migrated from issue #549) and the desktop verified-resolve coverage todo. +- Phase 42 added: API unpin integrity +- Phase 43 added: FUSE write durability +- Phase 44 added: IPNS conflict handling +- Phase 45 added: Desktop FUSE write-durability cleanup +- Phase 46 added 2026-06-15: Desktop FUSE data-loss bugs + replay hardening +- Phase 47 added 2026-06-15: SDK folder-state and publish-path consolidation +- Phase 48 added 2026-06-16: SDK self-bootstrap regression fix + shared-folder/metadata consolidation +- Phase 49 added 2026-06-18: Shared-folder intra-share move + useFolderNavigation unwrap consolidation +- Milestone v1.1 REOPENED 2026-06-19 with hardening block (Phases 50–55) +- Phase 56 added 2026-06-21: FUSE & IPNS Durability Hardening +- Phase 57 added 2026-06-21: API CID/Provider Hardening & Module Dedup +- Phase 58 added 2026-06-21: IPNS Signature-Verify Coverage +- Phase 59 added 2026-06-23: FUSE IPNS Verify/Publish Hardening & Cleanup +- Phase 60 added 2026-06-23: IPNS Verification Cross-Layer Closeout -- Desktop + API +- **v2.0 Phases 61–69 added 2026-06-27**: Metadata and Sharing Refactor — read key-chaining + rotation soundness + write-revocation + TEE contract rewrite + schema cutover + web/FUSE integration ### Open Concerns -- **main web-e2e is RED** (run 27587113911) since PR #498 merged — self-bootstrap `loadFolder` clobbers fresher folderTree state with a stale IPNS snapshot, failing `bin-restore-after-reload.spec.ts` + `full-workflow.spec.ts:6.6.2`. Blocks the staging E2E gate. Tracked as Phase 48 REQ-1 (P0). - 6 LOW-priority tech debt items remain from M2 audit: Settings URL param parsing, OCC coverage, addManyFiles atomicity, conflict telemetry, lazy rotation, desktop E2E (see `.planning/milestones/m2/m2-v1.0-production-MILESTONE-AUDIT.md`) -- Recovery tool subfolder recovery limited by IPNS DHT propagation (root-level fully operational; per-file IPNS records may not be resolvable if not propagated — architectural limitation, not a bug) +- Recovery tool subfolder recovery limited by IPNS DHT propagation (root-level fully operational; per-file IPNS records may not be resolvable if not propagated -- architectural limitation, not a bug) +- **v2.0 open questions** (to resolve during respective phases): + - Q1 (Phase 68): Co-writer offline during write-key rotation -- accept explicit re-fetch requirement or add grace/notification? + - Q2 (Phase 63): Rotation host for pure-web users -- is a long chunked multi-session web rotation acceptable for large revokes, or is desktop the only host? + - Q3 (Phases 65, 68, 69): Write-recipient-vs-owner sub-share authority -- when C (write recipient) deletes a node the owner independently sub-shared to D, who controls revocation of D? ### Pending Todos -**2026-06-27:** Captured Phase 39 D-02 data-safety gap (surfaced by the v1.1 milestone audit): the web app performs permanent/hard delete with no confirmation dialog when the vault default delete mode is `'permanent'` (`useFolderMutations.ts` `deleteWithBehavior`). See `2026-06-27-add-permanent-delete-confirmation-dialog-in-web-app.md`. +**2026-06-27:** Captured Phase 39 D-02 data-safety gap: web app performs permanent/hard delete with no confirmation dialog. See `2026-06-27-add-permanent-delete-confirmation-dialog-in-web-app.md`. -**2026-06-23:** Captured a high-severity storage/quota bug — bin delete + empty-bin never unpin content/version CIDs (client SDK `packages/sdk/src/bin/index.ts`), confirmed on staging (a user with empty vault+bin showing 442 MB quota used; 9 orphaned content CIDs = 439 MB still pinned). See `2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md`. Also captured the Phase 59 Finding F deferral as a Phase 60 carry-forward: `2026-06-23-phase60-ipns-first-publish-strict-equality-cutover.md` (resolves_phase: 60). +**2026-06-23:** Captured high-severity storage/quota bug -- bin delete + empty-bin never unpin content/version CIDs. See `2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md`. -See `/gsd:check-todos` for the full pending list. **2026-06-21:** filed resolved/superseded todos #5 (IPNS S1/S2/S3 → PR #529), #10 (refactor Tier-1/2 → PR #538), and the per-file-IPNS-conflict todo (folded into the PR #538 robustness todo) to `completed/`; re-captured the 14 Tier-3 refactor items as a new todo; grouped the FUSE/IPNS, API-CID, and IPNS-verify deferred-findings todos into new Phases 56–58. **2026-06-19:** todos #7 (useFolderNavigation consolidation) and #8 (shared-move re-encrypt) were verified already-resolved by Phase 49 (confirmed in live code) and moved to `completed/`; ten remaining tech-debt/security todos were grouped into the reopened v1.1 hardening block (Phases 50–55, see Roadmap Evolution). The four older feature todos (ERC-1271 wallet auth, CRDT IPNS inbox research, async search index, alternative MFA factors) and the GSD-tooling STATE regression (#10) remain unscheduled. _Historical:_ the desktop (6) and SDK (4) groups addressed by Phase 46 (merged) and Phase 47 (PR #494) were moved to `.planning/todos/completed/` on 2026-06-15 and their ROADMAP scope boxes checked. The architecture todo to give the SDK client the root IPNS key so it self-bootstraps/lazy-loads `folderTree` (root cause of the "Folder not loaded" class; bin-restore gap surfaced while combing the #494 fix) was completed 2026-06-16 in PR #498 (branch `feat/sdk-client-self-bootstrap-folder-tree`) and moved to `.planning/todos/completed/`; its follow-ups (delete the now-redundant web `ensureFolderRegistered`/`useFolderNavigation` unwrap paths once self-heal proves out; optional negative-cache) were captured as a new pending todo. Remaining pending still includes the route-shared-folder-writes follow-up — the lone folder-state mutation not consolidated by Phase 47. The v1.1 verification-ledger todo (phases 18/31/32 missing VERIFICATION.md) was completed 2026-06-19 — all three reports authored (goal-backward, adversarially spot-checked), PERF-01..04 closed (PERF-03 via accepted override), and the milestone audit verdict flipped to `passed` (66/66, 20/20); moved to `.planning/todos/completed/`. +See `/gsd:check-todos` for the full pending list. ### Resolved All M2 blockers resolved. See `.planning/milestones/m2/m2-v1.0-production-MILESTONE-AUDIT.md`. +All v1.1 requirements code-satisfied (77/77). See `.planning/milestones/v1.1-MILESTONE-AUDIT.md`. + ### Quick Tasks Completed | # | Description | Date | Commit | Directory | @@ -332,9 +339,9 @@ All M2 blockers resolved. See `.planning/milestones/m2/m2-v1.0-production-MILEST --- -Last activity: 2026-06-26 +Last activity: 2026-06-27 -Last session: 2026-06-24T01:45:06Z +Last session: 2026-06-27T00:01:29Z ## Decisions @@ -347,32 +354,15 @@ Last session: 2026-06-24T01:45:06Z - [Phase 59-03]: D.3 current_seq_for_cas replaced by direct if current_seq.is_none() guard (same error text) - [Phase 59-03]: E.1 VerifiedResolve::signature_verified field removed; was never read, only written - [Phase 59-03]: E.4 bytesToHex helper removed alongside the unused public_key/private_key vector fields -- [Phase 59-02]: VerifyError::Legacy carries { cid: String, sequence_number: String } from bind_verified — no second resolve_ipns in any Legacy arm (T-59-04 TOCTOU eliminated) +- [Phase 59-02]: VerifyError::Legacy carries { cid: String, sequence_number: String } from bind_verified -- no second resolve_ipns in any Legacy arm (T-59-04 TOCTOU eliminated) - [Phase 59-02]: Display for VerifyError::Legacy includes cid and seq: 'legacy record: all signature fields absent (cid={cid}, seq={sequence_number})' - [Phase 59-02]: events.rs synthetic VerifiedResolve keeps signature_verified: false until Finding E.1 (plan 03) removes the field -- [Phase ?]: deser_opt_string maps legacy empty-string file_meta_ipns_name to None; serde compat shim mandatory for pre-Phase-45 journal replay -- [Phase 45-04]: IpnsResolveOutcome lives in error.rs with #[derive(Debug)] only — not thiserror, it is an outcome not an error +- [Phase 45-04]: IpnsResolveOutcome lives in error.rs with #[derive(Debug)] only -- not thiserror, it is an outcome not an error - [Phase 45-04]: resolve_ipns_for_replay preserves both contains(not found) and contains(404) predicates to avoid classification regression -- [Phase ?]: Conditional use imports route replay to fuse->operations::implementation and winfsp->platform::windows::operations::implementation for publish_file_metadata -- [Phase ?]: folder_key_cache seeded with root key in replay_for_vault; resolve_folder_key_cached wrapper memoizes per replay call, never persisted or shared -- [Phase ?]: journal_helpers: helper takes &OpenFileHandle directly (open_files entry removed before call) -- [Phase ?]: journal_helpers: WinFsp write_gen read after write_generation bump; fuser uses result field captured before mutation -- [Phase ?]: journal_helpers: build_mkdir_journal_entry called after child inode inserted so build_folder_metadata sees new child -- [Phase ?]: Keep @internal on ensureFolderLoaded; call directly from web hook -- [Phase 49-03]: onMove wired for files only in folder-view ContextMenu; list-view synthetic items stay readOnly (T-49-09) -- [Phase 49-03]: currentFolderId for SharedMoveDialog derived from breadcrumbs last entry (not separate state) -- [Phase 49-03]: SharedMoveDialog lazy-loads subtree via enumerateSharedSubtree in useEffect gated on open && shareId -- [Phase 49-04]: batchMoveItemsHandler clearSelection called after runWrite completes (full success only) -- [Phase 49-04]: SharedFolderRow drop uses row's id/ipnsName as authoritative dest (ignores payload parentId per T-49-12) -- [Phase 49-04]: SelectionActionBar onDelete/onDownload are no-op stubs (REQ-6 scopes to move parity only) -- [Phase 49-05]: SharedMoveDialogPage.dialog() scoped via .move-dialog-folder-list filter (avoids collision with private MoveDialog) -- [Phase 49-05]: readContentViaEditor dispatches rightClickFolderItem vs rightClickItem based on instanceof check (shared vs private browser page) -- [Phase 49-05]: Alice decrypt assertion uses FileListPage (private vault view), not SharedFileBrowserPage — owner reads own files via vault browser - [Phase 51-01]: CAS ConflictException (409) check placed before S1 BadRequestException (400) sequence check so concurrent-modification signals remain authoritative; S1 reuses anti-rollback incomingParsed to avoid double parse -- [Phase 51-03]: resolve_folder_key_cached cache left as HashMap> (not Zeroizing); cache is short-lived, cleared on replay_for_vault drop — only BFS queue and get_folder_key return changed +- [Phase 51-03]: resolve_folder_key_cached cache left as HashMap> (not Zeroizing); cache is short-lived, cleared on replay_for_vault drop -- only BFS queue and get_folder_key return changed - [Phase 51-03]: verify_ipns_resolve_signature absent-fields path returns Ok(None) + warn (D-03), not error; verify gate in resolve_folder_key BFS only (not fetch_merge_publish_parent replay path) -- [Phase 51-04]: updateFolderMetadataAndPublish SKIP zeroing — all client.ts call sites pass live session keys from folderTree state reused across session lifetime; caller retains ownership (T-47-01 documented skip with guard test) -- [Phase ?]: CBOR import: cborg decode used in sdk-core; parseCborData from ipns unavailable +- [Phase 51-04]: updateFolderMetadataAndPublish SKIP zeroing -- all client.ts call sites pass live session keys from folderTree state reused across session lifetime; caller retains ownership (T-47-01 documented skip with guard test) - [Phase 60-01]: decode_ipns_cbor_validity companion fn chosen over 3-tuple return; all 9 FUSE Legacy arms folded to Invalid; manual RFC3339 parse with 5-min skew buffer (D-04/D-07) - [Phase 60-02]: D-02 all 9 first-publish producers unified to embed sequence 1; coordinator.record_publish updated to match; vault-settings.service.ts forward-publish increment path unchanged - [Phase 60-04]: All 9 FUSE Legacy arms were pre-folded by 60-01 (compiler-forced); Task 1 re-pointed imports and deleted verify.rs only @@ -380,11 +370,11 @@ Last session: 2026-06-24T01:45:06Z - [Phase 60-04]: registry.rs VerifyError::Invalid maps to SdkError::RegistryError (fail-closed) - [Phase 60-04]: prepopulate.rs and vault.rs: scoped per-operation fail-closed (D-09) - [Phase 60-06]: D-11 go decision: per-op verify cost (mean 0.105 ms) justifies a short-TTL cache; cache key = ipnsName + base64(recordBytes); TEE republish and resolve paths never populate cache -- [Phase ?]: [Phase 60-05]: D-03 first-publish gate changed from {0n,1n} to strict {1n} only; embedded-0 now returns 400 -- [Phase ?]: [Phase 60-05]: D-06 parseCachedRecord null-signedRecord path returns null; CID mismatch discards cached result -- [Phase ?]: [Phase 60-05]: D-06 withCachedPublicKey enrich and equal-seq signatureV2 enrich removed from resolveRecord -- [Phase ?]: [Phase 60-05]: api:generate NOT required; changes are internal service/codec logic with no OpenAPI surface change +- [Phase 60-05]: D-03 first-publish gate changed from {0n,1n} to strict {1n} only; embedded-0 now returns 400 +- [Phase 60-05]: D-06 parseCachedRecord null-signedRecord path returns null; CID mismatch discards cached result +- [Phase 60-05]: D-06 withCachedPublicKey enrich and equal-seq signatureV2 enrich removed from resolveRecord +- [Phase 60-05]: api:generate NOT required; changes are internal service/codec logic with no OpenAPI surface change ## Operator Next Steps -- Start the next milestone with /gsd-new-milestone +- Run `/gsd-plan-phase 61` to begin Phase 61: AAD-Bound Seal Primitive and Cross-Language KAT From efd9269ff662343c8445197338c282b4659a146c Mon Sep 17 00:00:00 2001 From: Michael Yankelev Date: Sat, 27 Jun 2026 02:38:38 +0200 Subject: [PATCH 4/5] docs: v2.0 architecture and pitfalls research Architecture + Pitfalls research for the node/v3 sharing refactor; archive v1.1 top-level research under _v1.1-archive/. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01SDNQVLoAw4DbPrPvtQPobh Entire-Checkpoint: 6c3c2756738d --- .planning/research/ARCHITECTURE.md | 1168 +++++------------ .planning/research/PITFALLS.md | 559 +++++--- .planning/research/SUMMARY.md | 206 +-- .../research/_v1.1-archive/ARCHITECTURE.md | 979 ++++++++++++++ .../research/{ => _v1.1-archive}/FEATURES.md | 0 .planning/research/_v1.1-archive/PITFALLS.md | 346 +++++ .../research/{ => _v1.1-archive}/STACK.md | 0 .planning/research/_v1.1-archive/SUMMARY.md | 187 +++ 8 files changed, 2197 insertions(+), 1248 deletions(-) create mode 100644 .planning/research/_v1.1-archive/ARCHITECTURE.md rename .planning/research/{ => _v1.1-archive}/FEATURES.md (100%) create mode 100644 .planning/research/_v1.1-archive/PITFALLS.md rename .planning/research/{ => _v1.1-archive}/STACK.md (100%) create mode 100644 .planning/research/_v1.1-archive/SUMMARY.md diff --git a/.planning/research/ARCHITECTURE.md b/.planning/research/ARCHITECTURE.md index 0f5c18f039..3cb0b3dfce 100644 --- a/.planning/research/ARCHITECTURE.md +++ b/.planning/research/ARCHITECTURE.md @@ -1,979 +1,397 @@ -# Architecture: IPFS Infrastructure v1.1 - -**Domain:** IPNS reliability, database minimization, BYO-IPFS, performance baselines -**Researched:** 2026-03-07 -**Confidence:** HIGH (existing codebase analyzed; Kubo APIs verified against official docs) - ---- - -## Table of Contents - -1. [Existing Architecture Summary](#1-existing-architecture-summary) -2. [IPNS Resolution: Replace delegated-ipfs.dev](#2-ipns-resolution-replace-delegated-ipfsdev) -3. [Database Minimization: Migration Path](#3-database-minimization-migration-path) -4. [BYO-IPFS Node Support](#4-byo-ipfs-node-support) -5. [Performance Instrumentation](#5-performance-instrumentation) -6. [Component Boundary Map](#6-component-boundary-map) -7. [Data Flow Changes](#7-data-flow-changes) -8. [Suggested Build Order](#8-suggested-build-order) -9. [Sources](#9-sources) - ---- - -## 1. Existing Architecture Summary - -### Current IPNS Flow (What Changes) +# Architecture: v2.0 Metadata and Sharing Refactor (node/v3 Integration) + +**Domain:** Brownfield integration of ratified node/v3 read key-chaining design into an existing 8-layer monorepo +**Researched:** 2026-06-27 +**Confidence:** HIGH — design is implementation-ready; all cited symbols verified against live code + +## 1. Design Source of Truth (not relitigated here) + +The design is complete and ratified. Sources: + +- `.planning/design/2026-06-26-sharing-read-keychaining-design.md` — canonical implementation spec +- `docs/adr/0001-write-revocation-full-ed25519-rotation.md` — write-revocation mechanism ratified as (c) +- `docs/adr/0002-read-revocation-protects-future-content-only.md` — read-revoke scope ratified +- `CONTEXT.md` — pinned terminology (`readKey`/`writeKey`, `generation`/`keyEpoch`/`sequenceNumber`, `grant`, `scope exit`) + +## 2. Existing System Architecture (verified) + +The live system has 8 layers: + +| Layer | Path | Role | +| ----- | ---- | ---- | +| crypto primitives (TS) | `packages/crypto/src/` | AES-GCM/CTR, ECIES, Ed25519, HKDF | +| core domain (TS) | `packages/core/src/` | Metadata schemas + codecs, IPNS record construction | +| sdk-core (TS) | `packages/sdk-core/src/` | Stateless upload/download/CRUD/IPNS | +| sdk (TS) | `packages/sdk/src/` | Stateful client: sharing, bin, invite | +| api (NestJS) | `apps/api/src/` | Untrusted relay: publish gate, share bookkeeping | +| tee-worker | `apps/tee-worker/src/` | Phala enclave: 6h IPNS republish batch signer | +| web | `apps/web/src/` | React 18 SPA, Zustand stores, Web Worker crypto | +| fuse/desktop (Rust) | `crates/fuse/src/`, `apps/desktop/` | FUSE/WinFsp virtual filesystem, Tauri shell | + +Rust mirrors of layers 1–3 live in `crates/crypto/`, `crates/core/`, `crates/sdk/`, `crates/api-client/`. + +## 3. What the Design Replaces vs Extends + +### 3.1 Deleted (confirmed symbols verified in codebase) + +| Symbol | Path | Reason | +| ------ | ---- | ------ | +| `FolderMetadata`, `FolderEntry`, `FolderChild`, `EncryptedFolderMetadata` | `packages/core/src/folder/types.ts:15-53` | Replaced by unified `Node` | +| `FileMetadata`, `FilePointer`, `VersionEntry` (current form), `EncryptedFileMetadata` | `packages/core/src/file/` | Replaced by unified `Node` with content self-seal | +| `ShareKey` entity, `share_keys` DB table | `apps/api/src/shares/entities/share-key.entity.ts:14` | Entire per-item key fan-out model deleted | +| `addShareKeys()` (line 337), `reWrapForRecipients()` (line 469) | `apps/web/src/services/share.service.ts` | Fan-out replaced by single `readDescriptorRef` | +| `executeLazyRotation()` | `apps/web/src/services/share.service.ts:602` | Replaced by `rotateReadFromNode()` | +| `spawn_file_meta_reencrypt()` | `crates/fuse/src/metadata.rs:777` | Content self-seal makes move-reencrypt unnecessary | +| `originalFolderKeyEncrypted` field + restore re-encrypt path | `packages/core/src/bin/types.ts:69`, `packages/sdk/src/bin/index.ts:497,688` | Bin restore becomes a pure re-link | +| `encryptedChildKeys` JSONB column on `share_invites` | `apps/api/src/shares/entities/share-invite.entity.ts:59` | Single root-key wrap replaces per-child fan-out | +| `folder_ipns.public_key` column | `apps/api/src/ipns/entities/folder-ipns.entity.ts:63-64` (nullable `Buffer \| null`) | Derivable from k51 name via `publicKeyFromIpnsName`; null for shared-folder rows caused two Phase-60 regressions | +| Dual-source columns on `ipns_republish_schedule` | `apps/api/src/republish/entities/republish-schedule.entity.ts:40-60` (`latestCid`, `sequenceNumber`, `encryptedIpnsKey`, `keyEpoch`) | Collapsed into `ipns_records` as sole source | + +Caller note on `spawn_file_meta_reencrypt` deletion: both call sites must be removed — `crates/fuse/src/write_ops/implementation/rename.rs:248` and `crates/fuse/src/platform/windows/write_ops.rs:1183`. The Windows path cannot compile on macOS; `Cargo Check & Test (Windows)` CI gate is authoritative. + +### 3.2 Renamed / Schema-Migrated + +| Old | New | Where | +| --- | --- | ----- | +| `folder_ipns` table | `ipns_records` table | `apps/api/src/ipns/entities/folder-ipns.entity.ts` entity rename; entity class rename to `IpnsRecord`; all repositories and service imports updated | +| `folderKey` / `fileKey` / `rootFolderKey` | `readKey` | Terminology: retired in all code identifiers per `CONTEXT.md` | +| `ShareGrant` type name | `grant` (concept), no separate type wrapper | `shares` table row is the grant | +| `readKeyEcies` field | `readDescriptorRef` | `shares` entity | +| vault blob: `encryptedRootFolderKey` | `ECIES(rootReadKey)` + `ECIES(rootWriteKey)` | `packages/core/src/vault/blob.ts` — vault blob re-designed to carry two keys | + +### 3.3 New Components + +| Component | Responsibility | Lives In | +| --------- | -------------- | -------- | +| `sealAesGcmAad` / `unsealAesGcmAad` | AAD-bound AES-256-GCM seal/unseal | `packages/crypto/src/aes/seal.ts` (additive) | +| `buildNodeAad()` | Canonical AAD builder — frozen byte encoding | `packages/crypto/src/aes/seal.ts` (TS); `crates/crypto/` (Rust twin) | +| Cross-language KAT fixture | Byte-identical vector asserted by both TS and Rust | `crates/crypto/tests/cross_language.rs` + `packages/crypto/__tests__/` | +| `Node` / `SealedChildRef` / `PublishedNode` types + codecs | Unified metadata model with two sealed bodies | `packages/core/src/node/` (new subdirectory) | +| `rotateReadFromNode()` | Resumable read-rotation engine with crash-safe frontier | `packages/sdk-core/src/` (named files, not barrel) | +| Durable generation high-water map (`{nodeId → highestGeneration}`) | M1 downgrade defense — fail-closed on generation regression | IndexedDB (web), sqlite-adjacent journal (FUSE/desktop) | +| Durable seq high-water map (`{nodeId → highestSeq}`) | Within-generation rollback defense | IndexedDB (web), FUSE journal adjacent | +| Rotation job record | Crash-safe frontier for `rotateReadFromNode` | IndexedDB (web), FUSE durable state | +| `verifySubtreeClean()` | O(items) read pass flagging dirty edges; resume entry point | `packages/sdk-core/src/` | +| Atomic publish CAS | `UPDATE ipns_records SET … WHERE sequenceNumber = :expected` | `apps/api/src/ipns/ipns.service.ts` (replace non-atomic `findOne → save`) | +| Tombstone state machine | `tombstoned` flag on `ipns_records`; publish-gate rejection; `410` resolve | `apps/api/src/ipns/` | +| TEE lease-renewer contract | Receive marshaled `signedRecord`, verify signature, extend EOL only; no CID origination, no seq increment | `apps/tee-worker/src/routes/republish.ts` (rewrite) | +| Server-side generation gate | Forward-only `generation` per node, mirrors sequence anti-rollback | `apps/api/src/ipns/ipns.service.ts` | +| Grant-root awareness in FUSE | Per-grant scope computation for `delete`/`rename`/`move` paths | `crates/fuse/src/write_ops/` | +| `Node` as Rust enum | `enum Node { Folder { children }, File { content }, Root { children } }` | `crates/core/src/` | + +### 3.4 Modified (Extended) — Key Paths + +| Component | What Changes | +| --------- | ------------ | +| `packages/crypto/src/aes/seal.ts` | Add `sealAesGcmAad`/`unsealAesGcmAad`/`buildNodeAad`; `sealAesGcm` stays for non-node uses | +| `packages/core/src/ipns/create-record.ts` | Clients still self-sign; no change to signing mechanics — only the vault init path changes keys | +| `apps/api/src/ipns/ipns.service.ts` | Atomic CAS publish; generation forward-only gate; tombstone check; `parseCachedRecord`-null fall-through made case-dependent | +| `apps/api/src/ipns/ipns.service.ts:226` | Key-possession-only check unchanged structurally; tombstone check added before it | +| `apps/api/src/shares/share-invite.service.ts` | Invite wraps single root `readKey`; claim re-wraps to claimer; delete `encryptedChildKeys` fan-out | +| `apps/api/src/republish/republish.service.ts:257` | `unenrollIpns` must also tombstone the `ipns_records` row, not only delete the schedule row | +| `crates/fuse/src/inode.rs:434,452,658,716` | ECIES unwrap of `folderKeyEncrypted`/`ipnsPrivateKey` per child → symmetric `unsealAesGcmAad` of `readKeySealed`/`writeKeySealed` | +| `crates/fuse/src/replay.rs:365` | Journal replay ECIES unwrap → symmetric chain unwrap | +| `crates/fuse/src/publish.rs:140` | `resolve_sequence_strict` extended with generation high-water check | +| `apps/web/src/services/share.service.ts` | `executeLazyRotation` → `rotateReadFromNode`; folderTree reconcile before publish | + +## 4. Data Flow Under node/v3 + +### 4.1 Node Publish Envelope (on IPFS) ```text -Client API delegated-ipfs.dev Kubo - | | | | - |-- sign IPNS record ---->| | | - | |-- upsert folder_ipns (DB) --->| | - | |-- PUT /routing/v1/ipns/:name ->| | - | |<-- 200 OK or 502 -------------| | - |<-- { sequenceNumber } --| | | - | | | | - |-- resolve IPNS -------->| | | - | |-- GET /routing/v1/ipns/:name ->| | - | |<-- record bytes or 502 --------| | - | |-- fallback: query folder_ipns --| | - |<-- { cid, seqNum } -----| | | -``` - -**Key problem:** `delegated-ipfs.dev` is the sole IPNS resolution path from the API. The DB-cached CID is the fallback, but this means the DB is doing double duty as both CID cache and the reliable resolution source. The external service adds latency (10s timeout, 3 retries with backoff) and has documented 502 reliability issues. - -### Current Database Tables (What Shrinks) - -| Table | Rows/User | Purpose | Can Migrate to IPFS? | -| ------------------------- | ------------------ | --------------------------------------------------------------------------------- | ------------------------------------------------------ | -| `users` | 1 | UUID, publicKey | NO -- identity anchor | -| `auth_methods` | 1-5 | Auth provider links | NO -- server-side auth | -| `refresh_tokens` | 1-3 | JWT sessions | NO -- server-side auth | -| `vaults` | 1 | encryptedRootFolderKey, encryptedRootIpnsPrivateKey, rootIpnsName, ownerPublicKey | PARTIALLY -- rootFolderKey to IPFS | -| `folder_ipns` | N folders+files | latestCid, sequenceNumber, encryptedIpnsPrivateKey, keyEpoch | YES -- but serves as CID cache and concurrency tracker | -| `pinned_cids` | N files | CID, sizeBytes (quota tracking) | NO -- server enforces quota | -| `ipns_republish_schedule` | N folders+files | TEE republish state | NO -- TEE orchestration is server-side | -| `shares` | N shares | Sharer, recipient, encryptedKey, revocation | FUTURE -- CRDT inbox research | -| `share_keys` | N file/folder keys | Per-item encrypted keys for shares | FUTURE -- CRDT inbox research | -| `share_invites` | sparse | Link sharing tokens | NO -- server-validated tokens | -| `device_approvals` | sparse | MFA device approval state | NO -- short-lived server state | -| `tee_key_state` | 1 | TEE epoch tracking | NO -- server-side TEE coordination | -| `tee_key_rotation_log` | sparse | Rotation audit trail | NO -- admin audit | - -### Current IPFS Provider Abstraction - -The existing `IpfsProvider` interface is minimal (3 methods): - -```typescript -interface IpfsProvider { - pinFile(data: Buffer, metadata?: Record): Promise<{ cid: string; size: number }>; - unpinFile(cid: string): Promise; - getFile(cid: string): Promise; -} -``` - -Only `LocalProvider` (Kubo) exists. The interface is injected via `IPFS_PROVIDER` DI token in `IpfsModule.forRootAsync()`, configured from env vars `IPFS_LOCAL_API_URL` and `IPFS_LOCAL_GATEWAY_URL`. - ---- - -## 2. IPNS Resolution: Replace delegated-ipfs.dev - -### 2.1 Recommendation: DB-First Resolution + Kubo DHT Verification - -**Strategy:** Invert the current resolution priority. Make the DB-cached CID the primary resolution source (it is already written synchronously on every publish). Use the self-hosted Kubo node's `/api/v0/name/resolve` endpoint for background DHT verification. Demote `delegated-ipfs.dev` to a disabled-by-default fallback. - -**Why this approach:** - -1. **Eliminates external dependency.** The self-hosted Kubo node already participates in the IPFS DHT. It can resolve IPNS names directly without delegated-ipfs.dev. -2. **DB cache is already the reliable source.** The `IpnsService.resolveRecord()` method (lines 290-350 of `ipns.service.ts`) already prefers DB cache when its sequence number is higher than the network result. Making DB-first explicit simplifies the architecture. -3. **Kubo DHT resolution has improved.** Kubo 0.38+ has sweep providers with 97% fewer DHT lookups for large-scale operations. IPNS TTL defaults dropped from 1 hour to 5 minutes in recent releases, so changes propagate faster. -4. **PubSub for same-node resolution.** Kubo supports `Ipns.UsePubsub` for near-instant IPNS record propagation between connected peers. When both the publishing client and the TEE republisher connect through the same Kubo node, PubSub makes resolution instantaneous. - -### 2.2 Architecture Change - -**Before (current):** - -```text -IpnsService.resolveRecord() - -> DelegatedRoutingClient.resolve() -> https://delegated-ipfs.dev (primary, 10s timeout) - -> folder_ipns DB query (fallback on 502/timeout) - -> Compare sequence numbers, return highest -``` - -**After:** - -```text -IpnsService.resolveRecord() - -> folder_ipns DB query (primary, <5ms) - -> Return DB result immediately to caller - -> Async: KuboIpnsClient.resolve() -> http://kubo:5001/api/v0/name/resolve (background) - -> If DHT has higher sequence number, update DB cache - -> If DHT fails, no-op (DB is authoritative for our records) -``` - -### 2.3 New Component: KuboIpnsClient - -Wraps Kubo's RPC API for IPNS operations: - -```typescript -// apps/api/src/ipns/kubo-ipns.client.ts -@Injectable() -export class KuboIpnsClient { - constructor(private readonly configService: ConfigService) { - this.kuboApiUrl = configService.get('IPFS_LOCAL_API_URL', 'http://localhost:5001'); - } - - /** - * Resolve IPNS name via Kubo DHT. - * POST /api/v0/name/resolve?arg=&dht-timeout=5s&nocache=true - * Returns the resolved /ipfs/ path or null if not found. - */ - async resolve(ipnsName: string): Promise<{ path: string } | null> { - /* ... */ - } - - /** - * Publish via Kubo's native IPNS publisher (for TEE republish path). - * Requires key imported into Kubo's keystore first. - * POST /api/v0/name/publish?arg=/ipfs/&key=&ttl=5m - */ - async publish(keyName: string, cid: string, options?: { ttl?: string }): Promise { - /* ... */ - } - - /** - * Import an Ed25519 private key into Kubo's keystore. - * POST /api/v0/key/import?arg=&format=libp2p-protobuf-cleartext - * Body: multipart/form-data with key bytes - */ - async importKey(keyName: string, privateKeyBytes: Uint8Array): Promise { - /* ... */ - } - - /** - * List keys in Kubo's keystore (for cleanup). - * POST /api/v0/key/list - */ - async listKeys(): Promise { - /* ... */ - } - - /** - * Remove a key from Kubo's keystore. - * POST /api/v0/key/rm?arg= - */ - async removeKey(keyName: string): Promise { - /* ... */ - } -} -``` - -### 2.4 Publishing Path: Keep Pre-Signed Records for Client Publishes - -The current publishing flow has clients sign IPNS records locally and send pre-signed bytes to the API. This is correct for zero-knowledge -- the server never has the unencrypted IPNS private key. This does NOT change. - -**For client publishes:** The API receives pre-signed record bytes via `POST /ipns/publish`. The API: - -1. Upserts the `folder_ipns` DB record (unchanged -- this is the reliable CID source) -2. Broadcasts the pre-signed record to the DHT via Kubo's `/api/v0/routing/put` (replaces delegated-ipfs.dev) -3. Falls back to delegated-ipfs.dev only if Kubo routing put fails and the fallback is enabled - -**For TEE republishes:** The existing flow is preserved: the TEE decrypts the IPNS private key in hardware, signs the IPNS record inside the TEE boundary, and returns the **signed record bytes** (not the raw key) to the API. The API then broadcasts the pre-signed record to the DHT via Kubo's `/api/v0/routing/put`, replacing delegated-ipfs.dev as the broadcast target. - -The IPNS private key never leaves the TEE boundary. This maintains the security invariant that the API server never handles raw IPNS private keys. - -**Rejected alternative (Kubo key import):** An optimization was considered where the TEE returns raw Ed25519 private key bytes for temporary import into Kubo's keystore (`/api/v0/key/import` → `/api/v0/name/publish` → `/api/v0/key/rm`). This was rejected because it would export the private key from the trusted boundary, undermining the TEE security model even if the key is only transiently present in Kubo's memory. - -### 2.5 Kubo Configuration Changes - -Enable on the self-hosted Kubo node: - -```json -{ - "Ipns": { - "UsePubsub": true - }, - "Routing": { - "Type": "auto" - } +PublishedNode { + schema: "node/v3" + kind: folder | file | root -- PLAINTEXT, AAD input + id: uuid -- PLAINTEXT, AAD input + generation: u32 -- PLAINTEXT, AAD input; anti-rollback witness + aeadVersion: 1 -- PLAINTEXT, primitive tag + + readSealed: base64 -- AES-256-GCM(read-body, key=readKey, aad=buildNodeAad(id,kind,gen,body)) + writeSealed: base64 | null -- AES-256-GCM(write-body, key=writeKey, aad=buildNodeAad(id,kind,gen,body)) } ``` -`Routing.Type: "auto"` (default) uses both DHT and any configured delegated routers. `Ipns.UsePubsub: true` enables near-instant resolution when both publisher and resolver subscribe to the same pubsub topic. This is especially useful for same-node operations (client publishes, then immediately resolves on the same Kubo instance). - -### 2.6 Impact on Recovery Tool - -The recovery tool (`apps/web/public/recovery.html`) currently resolves IPNS via `delegated-ipfs.dev` directly from the browser (no API needed -- the tool works offline from the CipherBox API). - -Updated resolution order for recovery: - -1. Try the CipherBox API's `/ipns/resolve` endpoint (DB-first + Kubo DHT) -2. Fall back to `delegated-ipfs.dev` only if the API is unreachable (true offline recovery) -3. The tool should accept a manual CID input as a last resort (user pastes CID from backup) - -### 2.7 Modified Files - -| File | Change | -| ----------------------------------------------- | ---------------------------------------------------------------- | -| `apps/api/src/ipns/kubo-ipns.client.ts` | NEW -- Kubo RPC client for IPNS operations | -| `apps/api/src/ipns/kubo-ipns.client.spec.ts` | NEW -- Unit tests | -| `apps/api/src/ipns/ipns.service.ts` | MODIFY `resolveRecord()` to DB-first with async DHT verification | -| `apps/api/src/ipns/delegated-routing.client.ts` | MODIFY -- Add config flag to disable, demote to fallback | -| `apps/api/src/ipns/ipns.module.ts` | MODIFY -- Register `KuboIpnsClient` | -| `apps/api/src/republish/republish.service.ts` | MODIFY -- Option to use Kubo native publish for TEE path | -| `apps/api/.env.example` | ADD `IPNS_DELEGATED_ROUTING_ENABLED=false` | -| `apps/web/public/recovery.html` | MODIFY -- Updated resolution order | - ---- - -## 3. Database Minimization: Migration Path - -### 3.1 Table-by-Table Analysis - -#### 3.1.1 `vaults.encryptedRootFolderKey` -- MIGRATE TO IPFS +`buildNodeAad` encodes: `"cipherbox/node-seal/v1" ‖ 0x00 ‖ nodeId(16B raw UUID bytes) ‖ kind(1B: 0x01 folder/0x02 file/0x03 root) ‖ generation(4B BE) ‖ role(1B: 0x01 body/0x02 child-readkey/0x03 content/0x04 child-writekey)`. Byte encoding frozen — the KAT pins it. -**Current state:** The `vaults` table stores `encryptedRootFolderKey` (ECIES-wrapped with user's publicKey) and `encryptedRootIpnsPrivateKey` (also ECIES-wrapped, but redundant since the IPNS key is HKDF-derivable). - -**Migration:** Move `encryptedRootFolderKey` into the IPFS blob pointed at by the root vault IPNS record. The blob currently contains only AES-GCM encrypted folder metadata. The new format prepends the ECIES-wrapped root folder key. - -**New blob format (version 2):** - -```text -Vault IPFS Blob v2: - Byte 0: version = 0x02 - Bytes 1-2: encryptedRootFolderKey length (uint16, big-endian) - Bytes 3..N: ECIES-encrypted rootFolderKey - Bytes N+1..: AES-GCM encrypted folder metadata (unchanged) - -Vault IPFS Blob v1 (current, no version byte): - All bytes: AES-GCM encrypted folder metadata -``` - -Detection: If the first byte is `0x02`, parse as v2. If the first bytes are a valid AES-GCM ciphertext (IV + ciphertext + tag), parse as v1. - -**Login flow change:** - -Before: +### 4.2 Key Unwrap Walk (replaces per-child ECIES) ```text -Login -> API GET /vault -> { encryptedRootFolderKey, rootIpnsName } - -> Derive IPNS key via HKDF - -> Resolve IPNS -> fetch blob -> decrypt metadata with rootFolderKey +Owner (or grantee) holds: + share-root readKey ← ECIES-unwrap(grant.readDescriptorRef, recipientPrivKey) [1 ECIES once] + +To navigate to a depth-d child: + for each level: + unseal parent read-body with parent.readKey + buildNodeAad(parentId, kind, gen, body) + for target child in SealedChildRef[]: + child.readKey = unsealAesGcmAad(child.readKeySealed, parent.readKey, + buildNodeAad(childId, child.kind, child.generation, child-readkey)) + resolve child.ipnsName → child envelope; verify generation ≥ high-water ``` -After: +This replaces all `cipherbox_crypto::ecies::unwrap_key` calls in `crates/fuse/src/inode.rs:434,452,658,716` and `crates/fuse/src/replay.rs:365`. -```text -Login -> Derive IPNS key via HKDF (no API call needed for key material) - -> API GET /ipns/resolve?ipnsName= -> { cid } - -> API GET /ipfs/ -> blob v2 - -> Extract encryptedRootFolderKey from blob header - -> ECIES decrypt rootFolderKey with privateKey - -> Decrypt folder metadata with rootFolderKey -``` +### 4.3 AAD Byte Encoding — Cross-Language Parity Surface -**Migration strategy (dual-write, version-aware read):** +The byte encoding of `buildNodeAad` is the only TS↔Rust parity contract that is silent on failure (a mismatch causes `unsealAesGcmAad` to return `DecryptionError` with no indication of which language produced the wrong AAD). The cross-language KAT — one committed fixture asserted by both `packages/crypto/__tests__/` and `crates/crypto/tests/cross_language.rs` — is the sole guard. It must be the **first deliverable** in the crypto phase and must include role byte `0x04` (child-writekey). -1. **Write path:** Client publishes root metadata in blob v2 format (encryptedRootFolderKey in header). ALSO continues sending encryptedRootFolderKey in the vault init call for backward compatibility with older clients and the current recovery tool. -2. **Read path:** Client checks first byte of blob. v2 -> extract key from blob. v1 or unrecognizable -> fall back to `GET /vault` for the key. -3. **Cutover criteria:** When all active clients support blob v2 reading, the `GET /vault` endpoint can stop returning `encryptedRootFolderKey`. The column is kept in the DB as disaster recovery fallback but is no longer the primary source. +Critical encoding decisions pinned in the design (do not deviate): -#### 3.1.2 `vaults.encryptedRootIpnsPrivateKey` -- ELIMINATE +- `nodeId` = raw 16 RFC-4122 bytes, not a hash, not hex +- `kind` = `0x01/0x02/0x03` (not a string) +- `generation` = 4-byte big-endian +- `role` bytes = `0x01..0x04` +- Domain separator ends with `0x00` null byte before `nodeId` -This field is redundant. The root IPNS private key is deterministically derivable from the user's secp256k1 private key via HKDF: +### 4.4 Rotation Engine Data Flow ```text -rootIpnsPrivateKey = HKDF(userPrivateKey, info="cipherbox-root-ipns") -``` - -The client already performs this derivation. The server-stored copy was a bootstrapping convenience from v0.1 that is no longer needed. Stop sending it in vault init. Keep the DB column for backward compatibility but mark it deprecated. - -#### 3.1.3 `folder_ipns` -- KEEP BUT REDUCE ROLE - -The `folder_ipns` table currently serves four purposes: - -1. **CID cache** -- Reliable fallback when IPNS DHT resolution fails -2. **Sequence number tracking** -- Optimistic concurrency control (409 Conflict detection) -3. **Encrypted IPNS key storage** -- For TEE enrollment -4. **Record type metadata** -- folder vs file distinction - -With reliable Kubo IPNS resolution (section 2): - -- Purpose 1 (CID cache) is still valuable as a fast path (<5ms vs seconds for DHT), but no longer critical as the sole reliable source -- Purpose 2 (sequence numbers) is essential and CANNOT move to IPFS. You need to know the current sequence BEFORE publishing. This is inherently a server-side coordination concern. -- Purpose 3 (encrypted IPNS key) is duplicated in `ipns_republish_schedule`. Can be deduplicated by having the republish schedule reference `folder_ipns` instead of storing its own copy. Minor cleanup. -- Purpose 4 (record type) is metadata, could be derived but not worth the effort to remove. - -**Recommendation:** Keep `folder_ipns` as a "publish coordination table." Rename its conceptual role in documentation. Do NOT attempt to eliminate it -- sequence number tracking is a hard requirement for conflict detection. - -#### 3.1.4 `shares`, `share_keys` -- FUTURE (CRDT INBOX RESEARCH ONLY) - -Per the CRDT-based IPNS inbox todo, this milestone performs research only. The tables stay as-is. Key findings: - -- G-Set CRDT solves concurrent share additions (append-only inbox) -- Write-access control via signed envelopes prevents inbox spam -- Revocation by key rotation (not by modifying inbox) aligns with existing lazy revocation pattern -- State size growth needs compaction strategy -- **Dependency:** Requires reliable IPNS resolution (section 2) before inbox-based discovery is viable -- **Scope for v1.1:** Document the CRDT approach as a design RFC. Do NOT implement. - -#### 3.1.5 `device_approvals` -- MIGRATE TO IPFS (POSSIBLE BUT NOT RECOMMENDED) - -Device approvals are short-lived (15-minute expiry), require real-time status updates (pending -> approved/denied), and involve server-mediated notification polling. IPNS polling at 30s intervals is too slow for MFA approval UX. Keep server-side. - -#### 3.1.6 `pinned_cids` -- KEEP (QUOTA ENFORCEMENT) - -The server must enforce storage quotas. If quota tracking moves to IPFS, a malicious client could lie about their storage usage. The server must independently track pinned CID sizes. - -### 3.2 Post-Migration Database Role - -After v1.1 migrations, the database serves these purposes: - -| Purpose | Tables | Why Server-Side | -| -------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------- | -| Identity | `users`, `auth_methods` | Server-mediated auth | -| Sessions | `refresh_tokens` | JWT lifecycle | -| Quota enforcement | `pinned_cids` | Server must prevent quota abuse | -| Publish coordination | `folder_ipns` | Sequence numbers for concurrency | -| TEE orchestration | `ipns_republish_schedule`, `tee_key_state`, `tee_key_rotation_log` | Server schedules TEE work | -| Sharing graph | `shares`, `share_keys`, `share_invites` | Until CRDT inbox (v1.2+) | -| MFA | `device_approvals` | Short-lived approval state | -| Vault metadata | `vaults` (reduced) | ownerPublicKey, rootIpnsName, encryptedRootFolderKey (permanent fallback) | - -**What changed:** `encryptedRootIpnsPrivateKey` is deprecated (HKDF-derivable, redundant). `encryptedRootFolderKey` is retained as a permanent DB fallback even after migration to vault blob v2 on IPFS -- the IPFS copy is the primary source for recovery tool independence, but the DB copy ensures login works even if IPNS resolution fails. The server's role shifts from key escrow to coordination relay, though it still holds one ECIES-wrapped key for resilience. - ---- - -## 4. BYO-IPFS Node Support - -### 4.1 Recommendation: Server-Relay Model with Provider Abstraction - -**Decision:** Server-relay (not client-direct) because: - -1. **Quota tracking:** Server must see upload sizes to enforce quota. Client-direct bypasses quota enforcement entirely. -2. **IPNS publish coordination:** Sequence numbers are tracked server-side in `folder_ipns`. Bypassing the API means no conflict detection. -3. **CORS/connectivity:** Kubo's RPC API has no CORS headers by default. Client-direct from the browser requires the user's node to be publicly accessible or configured with CORS -- unreliable for home nodes behind NAT. -4. **Credential safety:** Server-relay means user's IPFS node credentials (API keys, auth tokens) are stored server-side, but the data passing through is already AES-256-GCM encrypted ciphertext. The server cannot read content regardless of relay model. - -**Future exception:** Desktop clients (Tauri) can pin directly to a user's local Kubo node because there are no CORS restrictions and the desktop app is trusted. This is a v1.2+ enhancement. - -### 4.2 Provider Abstraction Extension - -Extend the existing `IpfsProvider` with a per-user factory: - -```typescript -// apps/api/src/ipfs/providers/ipfs-provider.interface.ts -- ADDITIONS -export interface IpfsProviderFactory { - /** Get the appropriate IPFS provider for a given user */ - getProvider(userId: string): Promise; - /** Get the default (CipherBox-managed) provider */ - getDefaultProvider(): IpfsProvider; -} - -export const IPFS_PROVIDER_FACTORY = 'IPFS_PROVIDER_FACTORY'; +rotateReadFromNode(rootNodeId, reason, revokedRecipient?) → + 1. Root step: new readKey', generation' = gen+1, for files fileKey' (contentRekeyPending) + Re-seal root read-body under readKey' with buildNodeAad(…, generation') + Re-mint readDescriptorRef for all remaining recipients (HIGH-3: also re-mint any grants + rooted at any descendant — indexed query on shares.rootNodeId ∈ rotated set) + Delete revoked recipient's grant row + Publish root (CAS); update parent SealedChildRef.generation mirror + 2. Walk: per node, rotateOne(N, parentReadKey): + Re-fetch child list; merge any concurrent adds (HIGH-4 re-merge on 409) + New readKey'', generation'' = N.generation+1; for files fileKey'' + Re-seal; publish N (CAS); batch parent-link updates + 3. verifySubtreeClean(root): O(items) read pass; zero dirty edges = done + 4. Job record (IndexedDB/FUSE durable state): frontier+done per-node checkpoint ``` -The factory checks for per-user IPFS configuration. If none exists, it returns the default `LocalProvider`. If the user has a custom node configured, it returns a dual-pin provider that pins to both nodes. - -### 4.3 New Provider: UserCustomProvider - -```typescript -// apps/api/src/ipfs/providers/user-custom.provider.ts -export class UserCustomProvider implements IpfsProvider { - constructor( - private readonly endpoint: string, - private readonly authToken?: string, - private readonly providerType: 'kubo' | 'pinning-api' - ) {} - - async pinFile(data: Buffer): Promise<{ cid: string; size: number }> { - if (this.providerType === 'kubo') { - // Kubo RPC: POST /api/v0/add?pin=true&cid-version=1 - return this.pinViaKuboApi(data); - } else { - // IPFS Pinning Service API (spec: ipfs.github.io/pinning-services-api-spec) - // POST /pins with CID (requires content already available on IPFS network) - // For fresh uploads, use /api/v0/add equivalent or add+pin flow - return this.pinViaPinningApi(data); - } - } - - async unpinFile(cid: string): Promise { - /* provider-specific unpin */ - } - async getFile(cid: string): Promise { - /* provider-specific fetch */ - } -} -``` +Crash recovery: re-running `rotateOne` on an already-rotated node generates `readKey'''` and a second rotation step. This is safe — an extra rotation only strengthens the cut. -**Provider types supported:** - -| Provider Type | Protocol | Examples | Auth Method | -| ------------- | -------------------------- | ------------------------------ | ------------------------------ | -| `kubo` | Kubo RPC API (`/api/v0/*`) | Self-hosted Kubo, IPFS Desktop | None (localhost) or Basic Auth | -| `pinning-api` | IPFS Pinning Service API | Pinata, Filebase, web3.storage | Bearer token | - -### 4.4 Dual-Pin Provider - -The actual provider used for BYO-IPFS users wraps both the default and custom providers: - -```typescript -// apps/api/src/ipfs/providers/dual-pin.provider.ts -export class DualPinProvider implements IpfsProvider { - constructor( - private readonly defaultProvider: IpfsProvider, - private readonly customProvider: IpfsProvider - ) {} - - async pinFile(data: Buffer): Promise<{ cid: string; size: number }> { - // Always pin to default (CipherBox) node first -- this is the reliable source - const result = await this.defaultProvider.pinFile(data); - - // Best-effort pin to user's custom node - try { - await this.customProvider.pinFile(data); - } catch (error) { - // Log warning but do NOT fail the upload - logger.warn(`BYO-IPFS pin failed: ${error.message}`); - } - - return result; // Return CID from default node - } - - async getFile(cid: string): Promise { - // Try default node first, fall back to custom - try { - return await this.defaultProvider.getFile(cid); - } catch { - return await this.customProvider.getFile(cid); - } - } - - async unpinFile(cid: string): Promise { - // Only unpin from CipherBox's node (user manages their own retention) - await this.defaultProvider.unpinFile(cid); - } -} -``` +Convergence test: N is done iff `parent.SealedChildRef[N].generation == N.envelope.generation` and that generation exceeds the pre-job baseline. -### 4.5 User Settings Entity - -```sql -CREATE TABLE user_ipfs_config ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL UNIQUE REFERENCES users(id) ON DELETE CASCADE, - provider_type VARCHAR(20) NOT NULL DEFAULT 'default', -- 'default' | 'kubo' | 'pinning-api' - endpoint_url VARCHAR(500), - auth_token_encrypted BYTEA, -- encrypted with server-side key - created_at TIMESTAMP DEFAULT NOW(), - updated_at TIMESTAMP DEFAULT NOW() -); -``` +### 4.5 TEE Contract Change (Section 6 of design) -**`auth_token_encrypted`:** Encrypted with a server-side symmetric key (from env var), NOT the user's key. The server needs to decrypt it to authenticate with the user's IPFS node. This is acceptable because the data being relayed is already AES-256-GCM encrypted ciphertext -- the server cannot read it regardless. +Current TEE flow (broken — verified at `apps/tee-worker/src/routes/republish.ts:79`): receives `sequenceNumber`, increments to `+ 1n`, builds and signs a new record. -### 4.6 API Endpoints +New TEE contract: ```text -GET /vault/ipfs-config Get user's IPFS node configuration -PUT /vault/ipfs-config Set/update IPFS node configuration -DELETE /vault/ipfs-config Remove custom config (revert to default) -POST /vault/ipfs-config/test Test connectivity to user's IPFS node +API → TEE: send marshaled signedRecord (existing signed bytes) +TEE: parse record; verify Ed25519 signature + assert publicKeyFromIpnsName(ipnsName) == record.pubkey + derive currentEpoch from own clock (never from relay's scalar) + emit same record with same value (CID) and same sequence, only later EOL +TEE → API: re-signed record (no CID, no seq change); optional upgradedEncryptedKey if epoch migrated +API: UPDATE ipns_records SET signed_record = :new WHERE sequenceNumber = :loaded (idempotent CAS) ``` -The test endpoint attempts a small pin+unpin operation on the user's node to verify connectivity and auth. +The idempotent EOL-renewal CAS (`WHERE sequenceNumber = :loaded`) guarantees the renewal can never regress `latestCid`/`sequenceNumber`. The tombstone check must also reject tombstoned names presented to the TEE renewer. -### 4.7 Web UI: Settings Page Extension - -Add an "IPFS Node" section to the existing settings page: +### 4.6 Write-Revocation Cascade (ADR 0001 — Ed25519 rotation) ```text -[IPFS Storage] - Provider: ( ) CipherBox Default ( ) Custom Kubo Node ( ) Pinning Service - Endpoint: [http://my-node:5001 ] - Auth Token: [optional, for pinning services ] - [Test Connection] [Save Configuration] - - Status: Connected -- last verified 2 minutes ago +Per node in revoked-writer's scope: + generate new Ed25519 keypair → new k51 ipnsName' + generate new writeKey' + re-seal write-body (now carrying ed25519'.priv) under writeKey' + update parent SealedChildRef.ipnsName to ipnsName' + re-seal parent read-body (same readKey, new ipnsName reference) + publish new name (seq=1, fresh enroll); tombstone old name + TEE: unenroll old name, enroll new name + Update all grant rows: shares.rootIpnsName → new name; re-mint writeDescriptorRef for surviving co-writers ``` -### 4.8 IPNS and Conflict Detection Implications - -BYO-IPFS affects ONLY where encrypted data is pinned. It does NOT affect: - -- **IPNS publishing:** All publishes still go through the CipherBox API. The client signs IPNS records, the API broadcasts to the DHT. -- **Sequence numbers:** Still tracked in `folder_ipns`. Conflict detection is unchanged. -- **TEE republishing:** Unaffected -- TEE uses CipherBox's Kubo node. -- **IPNS resolution:** Unaffected -- IPNS resolves to a CID, and the CID is available from any node that has it pinned. +This is leaves-to-root (opposite of read-rotation which is root-first). -If a future version allows publishing directly to the user's node (bypassing the API), conflict detection would need to move client-side. This is NOT in v1.1 scope. +## 5. Build Order With Dependencies -### 4.9 Modified/New Files +The design's Section 7.2 build order is verified against the codebase. Dependency rationale follows each step. -| File | Change | -| -------------------------------------------------------------- | ---------------------------------------------------------- | -| `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` | ADD `IpfsProviderFactory` interface | -| `apps/api/src/ipfs/providers/user-custom.provider.ts` | NEW -- Custom Kubo/Pinning API provider | -| `apps/api/src/ipfs/providers/user-custom.provider.spec.ts` | NEW -- Unit tests | -| `apps/api/src/ipfs/providers/dual-pin.provider.ts` | NEW -- Dual-pin wrapper provider | -| `apps/api/src/ipfs/providers/dual-pin.provider.spec.ts` | NEW -- Unit tests | -| `apps/api/src/ipfs/providers/provider-factory.service.ts` | NEW -- Per-user provider resolution | -| `apps/api/src/ipfs/providers/provider-factory.service.spec.ts` | NEW -- Unit tests | -| `apps/api/src/ipfs/ipfs.module.ts` | MODIFY -- Register factory, user config repository | -| `apps/api/src/ipfs/ipfs.controller.ts` | MODIFY -- Use factory instead of direct provider injection | -| `apps/api/src/vault/entities/user-ipfs-config.entity.ts` | NEW -- User IPFS config entity | -| `apps/api/src/vault/dto/ipfs-config.dto.ts` | NEW -- IPFS config CRUD DTOs | -| `apps/api/src/vault/vault.controller.ts` | MODIFY -- Add IPFS config endpoints | -| `apps/api/src/migrations/XXXXXXXXX-AddUserIpfsConfig.ts` | NEW -- Create table migration | -| `apps/web/src/components/settings/IpfsNodeConfig.tsx` | NEW -- IPFS config UI | -| `apps/web/src/routes/SettingsPage.tsx` | MODIFY -- Add IPFS config section | +### Phase 1 — `packages/crypto`: AAD-Bound Seal Primitive + KAT ---- +**Files changed:** `packages/crypto/src/aes/seal.ts` (add `sealAesGcmAad`/`unsealAesGcmAad`/`buildNodeAad`), `packages/crypto/__tests__/` (KAT), `crates/crypto/` (Rust twin + cross-language test in `crates/crypto/tests/cross_language.rs`) -## 5. Performance Instrumentation +**Why first:** Self-contained; no consumer breaks. The frozen byte encoding must be committed before any consumer seals a `Node` — a retroactive encoding change would require rotating every sealed body. The KAT must exist before the core codec uses the primitives (otherwise byte-mismatch failures are silent decryption errors at FUSE). -### 5.1 Existing Infrastructure +**Dependency:** Nothing above this. -The API already has Prometheus metrics via `prom-client`: +### Phase 2 — `packages/core`: Unified `Node` Codec (Keystone) -- **Histogram:** `cipherbox_http_request_duration_seconds` with labels (method, route, status_code) and buckets [0.01 to 10s] -- **Counters:** file uploads/downloads/unpins, IPNS publishes/resolves, republish runs, auth logins -- **Gauges:** users total, files total, storage bytes, IPNS entries by type, republish schedule by status -- **Interceptor:** `HttpMetricsInterceptor` captures all HTTP request durations automatically -- **Endpoint:** `GET /metrics` exposes Prometheus-compatible output -- **Polling:** DB gauges refresh every 30 seconds +**Files changed:** Delete `packages/core/src/folder/`, `packages/core/src/file/metadata.ts` (core codec parts — keep IPNS derivation helpers). Add `packages/core/src/node/types.ts`, `packages/core/src/node/codec.ts`. Update `packages/core/src/vault/blob.ts` (two keys in vault blob). Update `packages/core/src/bin/types.ts` (delete `originalFolderKeyEncrypted`). Update `packages/core/src/index.ts` exports. -### 5.2 New Server-Side Metrics +**Why second:** Nothing below `packages/core` typechecks until `Node`/`SealedChildRef`/`PublishedNode` exist. `packages/sdk-core`, `packages/sdk`, `apps/web`, and all Rust crates import from this package. `dist/` must be rebuilt before any consumer. -Add domain-specific histograms for IPFS/IPNS latency tracking: +**Dependency:** Phase 1 (`sealAesGcmAad`/`buildNodeAad` are the only new crypto calls here). -```typescript -// IPNS resolution latency by source and outcome -readonly ipnsResolveDuration: client.Histogram; -// Labels: source (db|kubo_dht|delegated), result (hit|miss|error) -// Buckets: [0.005, 0.01, 0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10] +**Invariant to document in `METADATA_SCHEMAS.md`:** `generation` is authoritative only on the child's own published envelope; `SealedChildRef.generation` and `shares.rootGeneration` are convergence witnesses. `fileKey` is no longer ECIES-wrapped — it lives inside the sealed read-body (semantic type change, not a rename). -// IPNS publish latency by target and outcome -readonly ipnsPublishDuration: client.Histogram; -// Labels: target (db|kubo_dht|delegated), result (success|error) -// Buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10, 30] +### Phase 3 — `packages/sdk-core`: Read-Chain Navigation + Rotation Driver -// IPFS operation latency (pin/unpin/get) -readonly ipfsOperationDuration: client.Histogram; -// Labels: operation (pin|unpin|get), provider (default|byo), result (success|error) -// Buckets: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10] +**Files changed:** New named files (not barrel additions) for read-chain walk (`chainReadKey`, `navigateToNode`), `rotateReadFromNode` + `rotateOne` + `verifySubtreeClean`, durable high-water utilities. Rebuild `dist/` before consumers. -// TEE republish batch processing duration -readonly republishBatchDuration: client.Histogram; -// Labels: result (success|partial|error) -// Buckets: [0.5, 1, 2.5, 5, 10, 30, 60, 120] -``` +**Why third:** `sdk-core` is stateless; it consumes `packages/core` `Node` types and `packages/crypto` seal primitives. The rotation engine lives here because it is shared between the web (browser) and FUSE (via the Rust `crates/sdk` wrapper). Named files (not fat `index.ts` barrel) because `vitest` coverage excludes barrels — coverage drop would trip the 80% gate. -### 5.3 Client-Side Performance Utility +**Dependency:** Phase 2 (`Node` types), Phase 1 (`sealAesGcmAad`). -Add a lightweight timing utility for the web app: +### Phase 4 — `packages/sdk`: Write-Chain, Sharing, Bin, Invites -```typescript -// apps/web/src/lib/perf.ts -export function startTimer(label: string): () => number { - const start = performance.now(); - return () => { - const durationMs = performance.now() - start; - logger.debug(`[perf] ${label}: ${durationMs.toFixed(1)}ms`); - return durationMs; - }; -} -``` +**Files changed:** Rewrite `packages/sdk/src/share/shared-write.ts` (structured write-body, role `0x04` child-writekey). Delete `addShareKeys`/`reWrapForRecipients` from `packages/sdk/src/`. Rewrite `packages/sdk/src/bin/index.ts` (restore as pure re-link; delete re-encrypt path). Rewrite invite claim (`packages/sdk/src/share/invite.ts` or equivalent). -**Key operations to instrument:** - -| Operation | File | What to Measure | -| ----------------------------------- | ------------------------ | -------------------------------------- | -| IPNS resolve (API round trip) | `folder.service.ts` | API call latency | -| IPNS publish (API round trip) | `folder.service.ts` | API call + DB + DHT publish | -| File upload (encrypt + upload) | `upload.service.ts` | Full pipeline including encryption | -| File download (fetch + decrypt) | `download.service.ts` | Full pipeline including decryption | -| Folder metadata decrypt | `folder.service.ts` | AES-GCM decryption of metadata blob | -| AES-GCM encrypt throughput | `packages/crypto` | Encryption speed in MB/s | -| Full page load to interactive | `useAuth.ts` | Auth completion to first folder render | -| Folder navigation (click to render) | `useFolderNavigation.ts` | Navigate + resolve + decrypt + render | - -### 5.4 Baseline Collection Strategy - -1. **Instrument:** Add timing hooks with minimal overhead (no-op when not collecting) -2. **Collect baselines:** Run standardized workloads against staging environment -3. **Document:** Store baselines in `.planning/baselines/PERFORMANCE_BASELINES.md` with environment specs, dates, and methodology -4. **Automate (stretch goal):** Add a Playwright E2E test suite that measures key user journeys and asserts maximum latencies - -**Standardized workload scenarios:** - -| Scenario | Description | Target Metric | -| --------------------------------- | -------------------------------------------------- | ---------------------------- | -| Login flow | Auth -> vault init -> root resolve -> first render | Total wall time | -| Upload 1 MB | Encrypt -> API upload -> IPNS publish -> confirm | Total wall time | -| Upload 50 MB | Same as above, larger payload | Total wall time, memory peak | -| Download 1 MB | API fetch -> decrypt -> save | Total wall time | -| Navigate 3 levels deep | Resolve root -> subfolder -> subfolder | Total wall time | -| IPNS publish + resolve round trip | Publish -> immediately resolve same name | Latency | -| Search 100 files | Index lookup -> render results | Latency | -| Share a folder | Wrap key -> API call -> confirm | Total wall time | - -### 5.5 Modified/New Files - -| File | Change | -| ----------------------------------------------- | ---------------------------------------------- | -| `apps/api/src/metrics/metrics.service.ts` | ADD new histograms (4 new metrics) | -| `apps/api/src/ipns/ipns.service.ts` | ADD histogram observations in resolve/publish | -| `apps/api/src/ipfs/ipfs.controller.ts` | ADD histogram observations in pin/get/unpin | -| `apps/api/src/republish/republish.service.ts` | ADD histogram observation for batch processing | -| `apps/web/src/lib/perf.ts` | NEW -- Client-side performance timing utility | -| `apps/web/src/services/folder.service.ts` | ADD timing instrumentation | -| `apps/web/src/services/upload.service.ts` | ADD timing instrumentation | -| `apps/web/src/services/download.service.ts` | ADD timing instrumentation | -| `tests/e2e/tests/performance-baselines.spec.ts` | NEW -- Baseline collection E2E test (stretch) | - ---- - -## 6. Component Boundary Map - -### New Components +**Why fourth:** `packages/sdk` wraps `sdk-core` and adds state + sharing semantics. The share + bin + invite rewrites all depend on `Node` types (Phase 2) and the rotation driver (Phase 3). -```text -apps/api/src/ - ipns/ - kubo-ipns.client.ts NEW -- Kubo RPC client for IPNS - kubo-ipns.client.spec.ts NEW -- Unit tests - - ipfs/ - providers/ - user-custom.provider.ts NEW -- BYO-IPFS provider (Kubo + Pinning API) - user-custom.provider.spec.ts NEW -- Unit tests - dual-pin.provider.ts NEW -- Dual-pin wrapper - dual-pin.provider.spec.ts NEW -- Unit tests - provider-factory.service.ts NEW -- Per-user provider resolution - provider-factory.service.spec.ts NEW -- Unit tests - - vault/ - entities/ - user-ipfs-config.entity.ts NEW -- User IPFS node config entity - dto/ - ipfs-config.dto.ts NEW -- IPFS config CRUD DTOs - - migrations/ - XXXXXXXXX-AddUserIpfsConfig.ts NEW -- user_ipfs_config table - -apps/web/src/ - lib/ - perf.ts NEW -- Performance timing utility - - components/ - settings/ - IpfsNodeConfig.tsx NEW -- IPFS node config UI -``` +**Dependency:** Phase 3. -### Modified Components +### Phase 5 — `apps/api`: Schema + Publish Gate + Tombstone + Atomic CAS -```text -apps/api/src/ - ipns/ - ipns.service.ts MODIFY -- DB-first resolution, async DHT verify - ipns.module.ts MODIFY -- Register KuboIpnsClient - delegated-routing.client.ts MODIFY -- Add disable flag, demote to fallback - - ipfs/ - ipfs.module.ts MODIFY -- Use provider factory pattern - ipfs.controller.ts MODIFY -- Use factory, add perf instrumentation - - vault/ - vault.service.ts MODIFY -- Handle vault blob v2 format - vault.controller.ts MODIFY -- Add IPFS config endpoints - - republish/ - republish.service.ts MODIFY -- Optional Kubo native publish for TEE - - metrics/ - metrics.service.ts MODIFY -- Add 4 new histograms - -apps/web/src/ - services/ - folder.service.ts MODIFY -- Vault blob v2 reading, perf timers - upload.service.ts MODIFY -- Perf instrumentation - download.service.ts MODIFY -- Perf instrumentation - - routes/ - SettingsPage.tsx MODIFY -- Add IPFS node config section - - public/ - recovery.html MODIFY -- Vault blob v2, updated IPNS resolution - -packages/crypto/src/ - vault/ - types.ts MODIFY -- Add blob v2 type definitions - blob.ts NEW -- Blob v2 serialization/deserialization -``` +**Files changed:** -### Inter-Component Communication +- **TypeORM migration:** Delete `share_keys` table + entity (`apps/api/src/shares/entities/share-key.entity.ts`). Slim `shares` entity: add `readDescriptorRef`/`writeDescriptorRef`/`rootNodeId`/`rootIpnsName`/`rootGeneration`, remove old per-item key columns. Rename `folder_ipns` → `ipns_records` (entity class, table name, all repository references). Drop `folder_ipns.public_key` column. Collapse `ipns_republish_schedule` duplicated columns into `ipns_records` (or fold the schedule table). +- **`apps/api/src/ipns/ipns.service.ts`:** Atomic publish CAS (`UPDATE … WHERE sequenceNumber = :expected`); server-side generation forward-only gate; tombstone state check before key-possession gate at line 226; `parseCachedRecord`-null case-split (shared-folder null is expected → apply seq floor; `signedRecord` CID≠`latestCid` mismatch → fail closed). Fix TEE republish to use `ipns_records` as sole source (not schedule snapshot). +- **`apps/api/src/republish/republish.service.ts:257`:** `unenrollIpns` must tombstone `ipns_records` row, not only delete the schedule row. +- **`apps/api/src/shares/entities/share-invite.entity.ts`:** Remove `encryptedChildKeys` JSONB column; add `readDescriptorRef` for the single root-key wrap. +- **Run `pnpm api:generate`** and commit the regenerated `packages/api-client/src/generated/` alongside these changes (pre-commit `scripts/check-api-client.sh` enforces this). -```text -IpfsController --> ProviderFactory --> LocalProvider (default) - --> DualPinProvider --> LocalProvider (always) - --> UserCustomProvider (best-effort) - -IpnsController --> IpnsService --> folder_ipns DB (primary, fast path) - --> KuboIpnsClient (async DHT verification) - --> DelegatedRoutingClient (disabled-by-default fallback) - -RepublishService --> TeeService (signing) - --> KuboIpnsClient (Kubo native publish, preferred) - --> DelegatedRoutingClient (fallback if Kubo fails) - -MetricsService <-- IpnsService (resolve/publish duration) - <-- IpfsController (operation duration) - <-- RepublishService (batch duration) - <-- ProviderFactory (BYO-IPFS operation duration) -``` +**Why fifth:** API schema changes unblock TEE worker (which references `ipns_records`) and web/FUSE (which call the new API shape). The atomic CAS must land before any rotation work exercises the publish path. The `check-api-client.sh` pre-commit hook will reject a commit that changes API endpoints without also regenerating the client. ---- +**Dependency:** Phase 2 (entity types reference `Node` schema shape conceptually; the migration itself is independent but must not reference deleted types). -## 7. Data Flow Changes +### Phase 6 — `apps/tee-worker`: Lease-Renewer Contract Rewrite -### 7.1 IPNS Resolution (Changed) +**Files changed:** `apps/tee-worker/src/routes/republish.ts` — replace line 79 `+ 1n` increment with re-sign-same-sequence logic; add signature verification of incoming marshaled record; add internal epoch derivation (remove relay-supplied epoch scalar trust); add name↔key binding assertion (`publicKeyFromIpnsName(ipnsName) == pubkey(decryptedKey)`); add tombstone check (reject renewal of tombstoned name). -**Before:** +**Why sixth:** Depends on Phase 5 (`ipns_records` as sole source, tombstone column existing). The TEE republish round-trip E2E (`tests/sdk-e2e` or staging smoke) gates this phase. -```text -Client -> API /ipns/resolve - -> DelegatedRoutingClient.resolve() [10s timeout, 3 retries] - -> Parse IPNS record bytes - -> Compare with DB-cached CID (folder_ipns) - -> Return highest sequence number result -``` +**Dependency:** Phase 5 (DB schema; sole-source contract). -**After:** +### Phase 7 — `apps/web`: Replace Lazy Rotation, Add High-Water, folderTree Reconcile -```text -Client -> API /ipns/resolve - -> DB query folder_ipns WHERE ipnsName = ? [<5ms] - -> Return DB result immediately - -> Fire-and-forget: KuboIpnsClient.resolve() [5s DHT timeout] - -> If DHT has higher sequence number, update folder_ipns - -> If DHT fails, no-op -``` +**Files changed:** `apps/web/src/services/share.service.ts` — replace `executeLazyRotation` (line 602) with `rotateReadFromNode` driver call; delete `addShareKeys` (line 337) and `reWrapForRecipients` (line 469) from per-mutation fan-out paths. Add durable M1 generation high-water to IndexedDB (alongside existing device identity store at `apps/web/src/lib/device/identity.ts`). Add durable seq high-water. Add generation-regression check in IPNS resolve path. Enforce `folderTree` reconcile before rotation publishes (existing reconcile-before-publish discipline at `#489`/`#494`). -### 7.2 IPNS Publishing (Changed for TEE Path) +**Why seventh:** Web consumes the API (Phase 5) and sdk-core (Phase 3). The generation high-water is web-specific durable state (IndexedDB). The folderTree desync pattern is pre-existing risk — the M1 generation check must compose with the existing `sequenceNumber` reconcile-before-publish discipline, not replace it. -**Before (TEE republish):** +**Dependency:** Phase 3 (rotation driver), Phase 5 (API shape). -```text -RepublishService -> TeeService.republish() -> signed record bytes - -> DelegatedRoutingClient.publish() -> delegated-ipfs.dev -``` +### Phase 8 — `crates/fuse`: Rust Integration (FUSE + WinFsp) -**After (TEE republish, preferred path):** +**Files changed:** -```text -RepublishService -> TeeService.republish() -> signed record bytes - -> KuboIpnsClient.importKey() -> Kubo keystore - -> KuboIpnsClient.publish() -> Kubo DHT native - -> KuboIpnsClient.removeKey() -> cleanup - -> Fallback: DelegatedRoutingClient.publish() if Kubo fails -``` +- Replace all `cipherbox_crypto::ecies::unwrap_key` calls in `crates/fuse/src/inode.rs:434,452,658,716` and `crates/fuse/src/replay.rs:365` with `cipherbox_crypto::aes::unseal_aes_gcm_aad` symmetric unwrap. +- Delete `spawn_file_meta_reencrypt` from `crates/fuse/src/metadata.rs:777` and remove both call sites: `crates/fuse/src/write_ops/implementation/rename.rs:248` and `crates/fuse/src/platform/windows/write_ops.rs:1183`. +- Add grant-root awareness to `delete`/`rename`/`move` paths — FUSE already holds the mounted tree, so ancestry is cheap. +- Add durable generation + seq high-water (adjacent to existing write journal in `crates/fuse/src/cache.rs` or `journal_helpers.rs`). +- Replace `crates/core/src/` Rust types: `Node` as a real Rust enum (`enum Node { Folder { children: Vec }, File { content: SealedContent }, Root { children: Vec } }`), not a struct with `Option` fields. +- Add `crates/crypto/src/aes/` Rust twin of `buildNodeAad` + `sealAesGcmAad`/`unsealAesGcmAad`. +- Add `rotateReadFromNode` to `crates/fuse/src/` rotation paths. +- Strict-verify each rotation republish through the verified chokepoint, recovering Ed25519 pubkey from the k51 name via `publicKeyFromIpnsName` (never from the now-dropped `public_key` column). -### 7.3 Vault Access on Login (Changed) +**Why eighth:** All prior layers must land first. The FUSE crate is the most complex consumer — it implements both macOS/Linux FUSE and Windows WinFsp, it has the widest ECIES-to-symmetric conversion surface, and it needs grant-root awareness which is net-new logic that builds on the finalized `Node` schema (Phase 2) and the rotation engine (Phase 3). -**Before:** +**Critical:** `crates/fuse/src/platform/windows/write_ops.rs` and anything under `crates/fuse/src/platform/windows/` cannot compile on macOS. The `Cargo Check & Test (Windows)` CI gate is authoritative for this phase. Budget a CI round-trip. The `super::` vs `super::super::` nesting trap (from Phase 55 history) applies in the nested `pub mod implementation` structure; verify path imports in the Windows module explicitly. -```text -Client -> API GET /vault -> { encryptedRootFolderKey, rootIpnsName, teeKeys } - -> Derive IPNS key via HKDF - -> API GET /ipns/resolve -> { cid } - -> API GET /ipfs/ -> blob v1 (encrypted metadata only) - -> Decrypt metadata with rootFolderKey from vault response -``` - -**After (v1.1, with blob v2):** - -```text -Client -> API GET /vault -> { rootIpnsName, teeKeys, ownerPublicKey } - -> Derive IPNS key via HKDF - -> API GET /ipns/resolve -> { cid } - -> API GET /ipfs/ -> blob v2 - -> Parse blob: extract encryptedRootFolderKey header - -> ECIES decrypt rootFolderKey with privateKey - -> Decrypt metadata with rootFolderKey - -> Fallback: if blob is v1, GET /vault.encryptedRootFolderKey -``` - -### 7.4 BYO-IPFS Upload (New) - -```text -Client -> encrypt file -> API POST /ipfs/upload { file } - -> ProviderFactory.getProvider(userId) - -> If user has custom config: - -> DualPinProvider.pinFile(data) - -> LocalProvider.pinFile(data) // CipherBox node (always, reliable) - -> UserCustomProvider.pinFile(data) // User node (best-effort, logged) - -> If no custom config: - -> LocalProvider.pinFile(data) // CipherBox node only - -> VaultService.recordPin(userId, cid, size) // Quota tracking (unchanged) - -> return { cid, size } -``` - -### 7.5 BYO-IPFS Download (New Path) - -```text -Client -> API GET /ipfs/:cid - -> ProviderFactory.getProvider(userId) - -> If user has custom config: - -> DualPinProvider.getFile(cid) - -> Try LocalProvider.getFile(cid) // CipherBox node first - -> If not found: UserCustomProvider.getFile(cid) // Fall back to user node - -> If no custom config: - -> LocalProvider.getFile(cid) - -> return file buffer -``` +**Dependency:** Phases 1–7 (consumes all new types, all new API endpoints, all new crypto primitives). ---- +## 6. Cross-Cutting Integration Concerns -## 8. Suggested Build Order +### 6.1 TS↔Rust Parity Surface (AAD bytes) -### Phase 1: Performance Instrumentation +The `buildNodeAad` byte encoding is the only cross-language contract that fails silently. One KAT fixture — a hardcoded `(nodeId, kind, generation, role) → aad_bytes` vector — must be asserted by: -**Why first:** Zero risk to existing functionality. Purely additive. Establishes baselines BEFORE making any architectural changes. Without baselines, we cannot measure whether subsequent phases improve or regress performance. +- `packages/crypto/__tests__/build-node-aad.test.ts` +- `crates/crypto/tests/cross_language.rs` (Rust `#[test]`) -**Scope:** +Both must pass before any consumer is written. The fixture must include role `0x04` (child-writekey). -- Add 4 new Prometheus histograms to `MetricsService` -- Instrument `IpnsService.resolveRecord()` and `publishRecord()` with timing -- Instrument `IpfsController` upload/download/unpin with timing -- Instrument `RepublishService.processRepublishBatch()` with timing -- Create client-side `perf.ts` utility -- Instrument key web app operations (folder navigate, upload, download) -- Run standardized baseline collection against staging -- Document baselines in `.planning/baselines/` +### 6.2 Triplication Surface: web / sdk-core / FUSE+WinFsp -**Dependencies:** None. Uses existing `prom-client` and `MetricsService` infrastructure. -**Risk:** LOW. Additive instrumentation only. No behavioral changes. -**Estimated scope:** Small. ~10 modified files, no new entities or migrations. +Three independent consumers must implement byte-identical AAD handling: -### Phase 2: IPNS Resolution Improvement +| Consumer | Language | Durable state location | Rotation host | +| -------- | -------- | ---------------------- | ------------- | +| `apps/web` | TypeScript (Web Crypto) | IndexedDB | Client browser (long-lived tab) | +| `packages/sdk-core` | TypeScript | Caller-provided (web: IndexedDB; FUSE: journal) | Shared library | +| `crates/fuse` | Rust | FUSE write journal / adjacent sqlite-like store | Desktop process (preferred — long-lived, `PublishCoordinator`) | -**Why second:** Fixes the most critical reliability issue (delegated-ipfs.dev dependency). Required before Phase 3 because moving rootFolderKey to IPFS makes IPNS resolution a login-critical path. Baselines from Phase 1 enable before/after comparison. +The `packages/sdk-core` rotation driver (`rotateReadFromNode`) is shared between web and FUSE (the Rust `crates/sdk` wraps `sdk-core`). The durable high-water state is caller-provided; each consumer wires its own storage backend (IndexedDB or Rust journal). The convergence test and frontier representation are identical regardless of host. -**Scope:** +### 6.3 folderTree Reconcile-Before-Publish Discipline -- Implement `KuboIpnsClient` for native Kubo IPNS resolution -- Refactor `IpnsService.resolveRecord()` to DB-first with async DHT verification -- Make `DelegatedRoutingClient` optional with config flag (`IPNS_DELEGATED_ROUTING_ENABLED`) -- Configure Kubo with `Ipns.UsePubsub: true` -- Optionally update `RepublishService` to use Kubo native publish -- Update recovery tool IPNS resolution fallback chain -- Measure resolution latency against Phase 1 baselines +Existing project memory: `folderTree` in Zustand `useFolderStore` and the SDK client's `folderTree` can desync (the `#489`/`#494` "Folder not loaded" class). Under v3, this matters more: a wrong "don't rotate" (because the tree appeared to show no covering grant) is a silent missed revocation. The rule is: -**Dependencies:** Phase 1 (for before/after measurement). -**Risk:** MEDIUM. Changing the resolution path could cause resolution failures if Kubo DHT is slower than expected. Mitigated by: (1) DB-first strategy means the fast path is always local, (2) delegated routing remains as a disabled-by-default fallback, (3) existing retry logic in the delegated client. -**Estimated scope:** Medium. 1 new file, ~5 modified files, Kubo config change. +> Before any rotation publishes, reconcile `folderTree` against the current `sequenceNumber`. If the tree cannot be reconciled, **defer** the mutation — never skip rotation. -### Phase 3: Database Minimization (rootFolderKey to IPFS) +This is an extension of the existing discipline, not a replacement. The web `rotateReadFromNode` call site in `share.service.ts` must check reconciliation status before starting the walk. -**Why third:** Requires reliable IPNS resolution (Phase 2). The vault blob v2 format is a breaking metadata change that needs careful migration across web, desktop, and recovery tool. +### 6.4 Durable Client State: What Persists Where -**Scope:** +| State | Web | FUSE/Desktop | +| ----- | --- | ------------ | +| `{nodeId → highestGeneration}` (M1 high-water) | IndexedDB (new store, alongside device identity in `apps/web/src/lib/device/identity.ts`) | FUSE write journal or adjacent key-value file | +| `{nodeId → highestSeq}` (within-generation floor) | IndexedDB | FUSE journal | +| Rotation job record (`jobId`, `rootNodeId`, `frontier[]`, `done[]`, `status`) | IndexedDB | FUSE durable state | -- Define vault blob v2 format in `packages/crypto/src/vault/` -- Implement blob v2 serialization/deserialization -- Implement dual-write in web client (blob v2 on publish, DB on vault init) -- Implement version-aware blob reading (v2 -> extract key, v1 -> fall back to DB) -- Update vault init flow to make encryptedRootFolderKey optional -- Mark encryptedRootIpnsPrivateKey as deprecated -- Update recovery tool for blob v2 format -- Update desktop client blob parsing -- Migration testing: new user, existing user upgrade, recovery -- Update `docs/METADATA_SCHEMAS.md` per evolution protocol +The published IPNS records (not the job record) are the source of truth for convergence. The job record is advisory — it makes resume fast but losing it triggers a `verifySubtreeClean` rescan. -**Dependencies:** Phase 2 (IPNS must be reliable before making it login-critical). -**Risk:** MEDIUM-HIGH. Metadata format change affects all clients. Mitigated by: (1) dual-write ensures backward compatibility, (2) version detection allows graceful fallback, (3) DB column is never dropped -- kept as disaster recovery. -**Estimated scope:** Large. New type definitions, serialization code, changes in 3 clients (web, desktop, recovery). +### 6.5 API Client Regeneration Gate -### Phase 4: BYO-IPFS Node Support +The pre-commit hook `scripts/check-api-client.sh` enforces that `packages/api-client/src/generated/` is staged whenever `apps/api/src/` changes. Phase 5 (api) triggers this. Run `pnpm api:generate` after every API endpoint/DTO/controller change and commit the regenerated files on the same branch. Failing to do so blocks the CI pre-commit. -**Why last:** Largest surface area (new entity, new providers, new UI, new API endpoints). Benefits from stable IPNS resolution (Phase 2) and reduced DB dependency (Phase 3). Independent in design but benefits from the improved infrastructure. +### 6.6 Sequence Race Residual -**Scope:** +The atomic CAS (`UPDATE ipns_records … WHERE sequenceNumber = :expected`) is the primary serialization point. It is not a full distributed lock — two co-writers at different clients can still race. The `PublishCoordinator.get_lock(name)` in `crates/fuse/src/metadata.rs:342` serializes the job against the same client's concurrent writes. The CAS turns a silent clobber into a `409 → refetch → retry` loop. This residual is accepted per the design. -- Create `user_ipfs_config` entity and migration -- Implement `UserCustomProvider` with Kubo RPC and Pinning Service API support -- Implement `DualPinProvider` wrapper -- Implement `ProviderFactory` for per-user provider resolution -- Refactor `IpfsController` to use factory -- Add IPFS config CRUD endpoints to vault controller -- Add connection test endpoint -- Build IPFS node configuration UI in settings page -- Implement dual-pin strategy (CipherBox node always + user node best-effort) -- Add BYO-IPFS specific Prometheus metrics -- E2E test: configure custom node, upload, verify pinned +## 7. Symbol Drift Report (Design vs Live Code) -**Dependencies:** Phase 2 (reliable IPNS) and Phase 3 (reduced DB) are not strict blockers but provide a better foundation. -**Risk:** MEDIUM. New provider implementations need thorough testing against real IPFS nodes and pinning services. Connectivity to arbitrary user nodes introduces unpredictable failure modes. Mitigated by: (1) dual-pin always keeps a copy on CipherBox node, (2) best-effort approach for user node, (3) connection test endpoint validates before saving config. -**Estimated scope:** Large. New entity, migration, 4+ new files, UI component, multiple API endpoints. +All design-cited symbols verified. No drift found in file paths. Line number accuracy verified for material symbols: -### Build Order Summary +| Design Citation | Verified | +| --------------- | -------- | +| `executeLazyRotation` at `apps/web/src/services/share.service.ts:602` | Line 602 confirmed | +| `revokeShare` keeping `ShareKey` rows "for lazy rotation" at `shares.service.ts:256-269` | Line 256 comment "kept for lazy rotation" confirmed | +| `addShareKeys` at `share.service.ts:337`, `reWrapForRecipients` at `:469` | Lines 337 and 469 confirmed | +| `ipns.service.ts:226` — "no ownership/share check" comment | Confirmed: comment at line 226 | +| `shared-write.ts:138-141,311` ECIES-wraps Ed25519 key | `ipnsPrivateKeyEncrypted` is hex-wrapped at line 137; `ipnsPrivateKey` at 277/303/361/392 — confirmed | +| `spawn_file_meta_reencrypt` at `crates/fuse/src/metadata.rs:777` | Line 777 confirmed | +| Callers: `rename.rs:248` and `platform/windows/write_ops.rs:1183` | Lines 248 and 1183 confirmed | +| `inode.rs:434,452` ECIES unwrap folder key / IPNS key | Lines 434 and 452 confirmed | +| `inode.rs:658,716` additional ECIES unwrap calls | Lines 658 and 716 confirmed | +| `replay.rs:365` ECIES unwrap | Line 365 confirmed | +| `folder_ipns.public_key` nullable `Buffer \| null` at `folder-ipns.entity.ts:63-64` | Confirmed | +| `republish-schedule.entity.ts:39-60` duplicated columns | `encryptedIpnsKey:40`, `keyEpoch:47`, `latestCid:53`, `sequenceNumber:60` confirmed | +| `republish.service.ts:257` `unenrollIpns` only deletes schedule row | Line 257 `scheduleRepository.delete` confirmed — no tombstone | +| `apps/tee-worker/src/routes/republish.ts:79` does `+ 1n` | Line 79 `+ 1n` confirmed | +| `tee.service.ts:110` batch republish | Line 110 `async republish` confirmed | +| `packages/core/src/ipns/create-record.ts` client self-signs | `createIpnsRecord` at line 31 confirmed | +| `publish.rs:140` `resolve_sequence_strict` — sequence only, no generation | Line 140 confirmed; `verified.sequence_number` only, no generation field | +| `share-invite.entity.ts` `encryptedChildKeys` JSONB column | Line 59 confirmed | +| `originalFolderKeyEncrypted` in bin types and re-encrypt path | `packages/core/src/bin/types.ts:69` + `packages/sdk/src/bin/index.ts:497,688` confirmed | -```text -Phase 1: Performance Instrumentation - [No dependencies, zero-risk baseline establishment] - | - v -Phase 2: IPNS Resolution Improvement - [Eliminates delegated-ipfs.dev, enables Phase 3] - | - v -Phase 3: Database Minimization - [Moves rootFolderKey to IPFS, requires reliable IPNS from Phase 2] - | - v -Phase 4: BYO-IPFS Node Support - [New capability, benefits from improved infrastructure] -``` +One notation note: the design cites `packages/core/src/file/metadata.ts:232` for `decryptFileMetadata` taking parent `folderKey`. The actual file has `decryptFileMetadata` at approximately line 231 (the comment "32-byte AES key of the parent folder" at line 227 confirms the semantic). Off by one from line 232; not material. -**Critical dependency:** Phase 2 MUST precede Phase 3. Moving rootFolderKey to IPFS without reliable IPNS resolution means users cannot log in if Kubo DHT is slow. +## 8. Phase Seam Recommendations for the Roadmap -**Parallel opportunity:** Phase 4's design and provider implementation can be developed in parallel with Phase 3's blob format work, but integration testing should wait for Phase 3 completion. +The 8-phase build order above maps directly to natural milestone phases. Suggested seaming: ---- +1. **Crypto primitive** (Phase 1) — shippable independently; no consumer breaks; KAT is the merge gate. +2. **Core keystone** (Phase 2) — nothing typechecks below until done; must rebuild `dist/` before proceeding. +3. **sdk-core rotation engine** (Phase 3) — rotation tests (`verifySubtreeClean`, crash-safety suite) gate this phase. +4. **sdk write/bin/invite** (Phase 4) — sdk E2E is the gate; the only real client→API IPNS publish/resolve round-trip. +5. **API schema + publish gate** (Phase 5) — DB migration + atomic CAS + tombstone + `api:generate`. All downstream consumers blocked until this lands. +6. **TEE worker** (Phase 6) — lease-renewer contract; republish E2E or staging smoke gate. +7. **Web** (Phase 7) — rotation UX + M1 high-water + reconcile discipline; Playwright E2E gates. +8. **FUSE + WinFsp** (Phase 8) — widest blast radius; Windows CI round-trip mandatory; grant-root awareness is net-new logic. -## 9. Sources +Phases that likely need deeper sub-phase research: Phase 8 (FUSE/WinFsp grant-root scope computation is novel; the Windows path can only be CI-verified). Phase 5 (TypeORM migration for `folder_ipns → ipns_records` rename with active FK references from `ipns_republish_schedule`, `shares`, `vaults` tables — verify all FK constraints before the migration). -### HIGH Confidence (Official Documentation, Verified) +## 9. Accepted Residuals (Not Fixed By This Refactor) -- [Kubo RPC API v0 Reference](https://docs.ipfs.tech/reference/kubo/rpc/) -- `/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import` endpoints verified against v0.40.0 -- [Kubo Configuration Reference](https://github.com/ipfs/kubo/blob/master/docs/config.md) -- `Ipns.UsePubsub`, `Routing.Type` configuration -- [IPFS Pinning Service API Spec](https://ipfs.github.io/pinning-services-api-spec/) -- Vendor-agnostic pinning API standard (OpenAPI spec) -- [IPNS Concepts](https://docs.ipfs.tech/concepts/ipns/) -- IPNS TTL default (5 minutes), PubSub resolution -- [prom-client GitHub](https://github.com/siimon/prom-client) -- Histogram, Counter, Gauge APIs +Per ADR 0002: already-distributed CIDs and prior file versions remain readable by anyone who held the `readKey`. The `contentRekeyPending` marker and optional per-file "re-encrypt now" operation are the high-sensitivity mitigation path. -### MEDIUM Confidence (Multiple Sources Agreeing) +Per ADR 0001 and design Section 9.2 open question 3: when a write-recipient (C) deletes a node that the owner independently sub-shared to a third party (D), C can unlink immediately but cannot revoke D's grant — the authority is split. Resolution (accept option (a) or (c) from the design) deferred to the implementing phase. -- [Kubo v0.38+ Release Notes](https://github.com/ipfs/kubo/releases) -- Sweep provider with 97% fewer DHT lookups, IPNS TTL changes -- [Kubo Issue #10484: Download & Upload IPNS Records](https://github.com/ipfs/kubo/issues/10484) -- Confirms `/api/v0/routing/put` for pre-signed records -- [Kubo Issue #8542: Publish IPNS with Signature](https://github.com/ipfs/kubo/issues/8542) -- Confirms key import workflow -- [Kubo Delegated Routing Docs](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md) -- `put-ipns` and `get-ipns` routing methods -- [IP Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- DHT improvements context +The colluding-relay residual: a malicious relay that drops rotation publishes and serves stale records is bounded by the durable client generation + seq high-water floors (M1 + Section 6.5), not eliminated. This is the explicit systemic residual. -### Codebase Analysis (Direct Code Review) +## Sources -- `apps/api/src/ipns/ipns.service.ts` -- Current resolution logic, DB fallback (resolveRecord lines 290-350) -- `apps/api/src/ipns/delegated-routing.client.ts` -- Current publish/resolve implementation (3 retries, 10s timeout) -- `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` -- Existing 3-method provider interface -- `apps/api/src/ipfs/providers/local.provider.ts` -- Current Kubo provider (pin/unpin/get via RPC API) -- `apps/api/src/ipfs/ipfs.module.ts` -- Current DI config (env var driven, single provider) -- `apps/api/src/ipfs/ipfs.controller.ts` -- Current upload/download/unpin with metrics -- `apps/api/src/vault/vault.service.ts` -- Current vault init with encryptedRootFolderKey -- `apps/api/src/vault/entities/vault.entity.ts` -- Current vault schema (6 columns) -- `apps/api/src/metrics/metrics.service.ts` -- Existing Prometheus infrastructure (gauges, counters, histogram) -- `apps/api/src/metrics/http-metrics.interceptor.ts` -- Existing HTTP duration tracking -- `apps/api/src/republish/republish.service.ts` -- Current TEE republish via delegated routing -- `apps/api/src/ipns/entities/folder-ipns.entity.ts` -- CID cache + sequence tracking -- `apps/api/src/republish/republish-schedule.entity.ts` -- TEE scheduling state -- All 13 database entities analyzed for migration feasibility +- `.planning/design/2026-06-26-sharing-read-keychaining-design.md` — design source of truth (Sections 2–7) +- `.planning/design/2026-06-26-sharing-flows-walkthrough.md` — flow-by-flow trace (Flows 1–8) +- `docs/adr/0001-write-revocation-full-ed25519-rotation.md` — write-revocation rationale +- `docs/adr/0002-read-revocation-protects-future-content-only.md` — read-revocation scope +- `CONTEXT.md` — pinned terminology +- `docs/ARCHITECTURE.md` — existing system overview +- `docs/METADATA_SCHEMAS.md` v1.1 — schemas being replaced +- Live codebase grep verification (all cited symbols confirmed as described above) diff --git a/.planning/research/PITFALLS.md b/.planning/research/PITFALLS.md index 0c3adf55b2..6400f5f473 100644 --- a/.planning/research/PITFALLS.md +++ b/.planning/research/PITFALLS.md @@ -1,346 +1,487 @@ -# Pitfalls Research +# Pitfalls Research — v2.0 Metadata and Sharing Refactor (node/v3) -**Domain:** IPFS infrastructure improvements -- replacing delegated routing, migrating DB state to IPFS/IPNS, BYO-IPFS node support, and performance baselines for an existing zero-knowledge encrypted storage app -**Researched:** 2026-03-07 -**Confidence:** HIGH for routing/migration pitfalls (verified against codebase + IPFS docs), MEDIUM for BYO-IPFS (fewer production precedents), MEDIUM for instrumentation (well-understood domain but CipherBox-specific interactions need validation) - -**Context:** CipherBox v1.0 shipped 2026-03-05 with 423K lines of TypeScript + Rust across 698 source files. IPNS is already deeply integrated: the `folder_ipns` table tracks per-folder and per-file IPNS records with sequence numbers for optimistic concurrency, the `ipns_republish_schedule` table drives TEE republishing every 6 hours, and `delegated-ipfs.dev` is the sole external routing provider (known unreliable, with DB-cached CID fallback). The database stores auth, vault keys, shares, device approvals, pinned CIDs for quota tracking, and all IPNS state. The recovery tool (`recovery.html`) directly resolves IPNS via `delegated-ipfs.dev` without any API intermediary. +**Domain:** Brownfield security/crypto refactor — read key-chaining, resumable read-rotation, write-revocation via full Ed25519 rotation, TEE/resolve contract rewrite +**Researched:** 2026-06-27 +**Confidence:** HIGH (sourced from design document, ADRs, codebase scar tissue, and project history) --- ## Critical Pitfalls -Mistakes that cause data loss, break existing functionality, or require rearchitecting completed features. +### Pitfall 1: AAD Byte-Encoding Drift Between TS and Rust = Silent Total Decryption Failure + +**What goes wrong:** +The `buildNodeAad` function must produce an identical byte sequence in TypeScript (`packages/crypto`) and Rust (`crates/crypto`). If either side differs in any encoding detail — UUID bytes as a string vs raw 16-byte RFC-4122 big-endian, `generation` as little-endian vs big-endian, `kind` byte value, `role` byte value, or the null separator after the domain string — then every cross-language unseal silently returns a `DOMException: The operation failed` with no indication of which field diverged. This is a **total decryption failure** across every Node sealed by one side and read by the other (web seals, FUSE reads; desktop seals, web reads). + +**Why it happens:** +The two crypto stacks share no code. TS uses `TextEncoder` for the domain string, Rust uses `b"..."` literals. UUID fields are easy to accidentally encode differently (`uuid.as_bytes()` = 16 raw bytes in RFC-4122 field order vs `uuid.to_string()` encoded as UTF-8). Generation is 4 bytes and big-endian is specified, but a developer implementing from spec can silently use the platform default. Role bytes must match the exact table (`0x01 body`, `0x02 child-readkey`, `0x03 content`, `0x04 child-writekey`); a transposition is invisible until an unseal fails. + +**How to avoid:** +The design mandates a committed cross-language Known-Answer Test (KAT) fixture — one hardcoded test vector committed in `crates/crypto/tests/cross_language.rs` AND asserted by `packages/crypto/__tests__`. This must be the **first deliverable** in the crypto primitive phase, before any consumer is built. The KAT must exercise all four role bytes. The AAD encoding must be frozen in writing (the design's Section 2.5 encoding table is that freeze) and both implementors must implement strictly from it, not from the other language's source. + +**Warning signs:** +Any `unsealAesGcmAad` call that worked in isolation (same-language round-trip) fails when the ciphertext crosses the TS/Rust boundary. FUSE read returning a deserialization error on a freshly-created node from the web app. KAT test missing or passing on mocked data. + +**Phase to address:** +Crypto primitive phase (Phase 1 / `packages/crypto`). The KAT must be committed and passing before any downstream phase begins. Every subsequent phase that adds a new `role` byte must extend the KAT fixture. --- -### Pitfall 1: Sequence Number Divergence When Switching Routing Providers +### Pitfall 2: CRIT-1 — Content-Key Rotation Omitted from rotateOne = Silent Read-Revocation Bypass **What goes wrong:** -The system currently has two sources of truth for IPNS sequence numbers: the `folder_ipns` PostgreSQL table and the DHT/delegated routing network. When replacing `delegated-ipfs.dev` with a self-hosted Someguy instance or Kubo DHT, the new routing endpoint may return different sequence numbers than the DB cache for a transition period. The `resolveRecord()` method in `ipns.service.ts` (line 290-350) already compares DB and network sequence numbers, taking the higher one. But during the switch, the new provider may have _no_ records (it has not seen previous publishes), causing resolution to return null from the network while the DB has valid data. If the resolution logic treats "null from network" differently than "null from delegated-ipfs.dev" -- or if the new provider starts returning stale records it picked up from DHT propagation with lower sequence numbers -- the system could serve outdated metadata, causing the client to see old folder contents or triggering false conflict detection (409 Conflict). - -The TEE republishing service (`republish.service.ts`) further complicates this: it publishes signed records to delegated routing and then syncs the sequence number back to `folder_ipns`. If the old and new routing providers are both receiving publishes during a transition window, the sequence number in each may diverge independently. +Re-sealing a file node's read-body under a new `readKey` is not sufficient. The revoked reader already cached the old `readKey`, used it to unseal `content`, and holds the raw `content.fileKey`. If the next file version is encrypted under the same `fileKey`, the revoked reader decrypts it the moment they obtain the new CID (from any IPFS gateway — they do not need IPNS). The revocation appears complete (entry-point navigation is cut) but file content remains accessible. **Why it happens:** - -- Developers assume a clean cutover is possible -- swap one URL for another -- IPNS records propagate through the DHT with eventual consistency; a new Someguy instance must discover existing records through DHT lookups, which takes time -- The TEE republisher runs on a 6-hour cycle and may publish to the old provider after the API has switched to the new one, creating a split-publish window +`rotateOne` is implemented to re-seal the read-body and bump `generation`; the file content path is a separate concern that is easy to miss when writing the rotation walk. `contentRekeyPending` must be set on the node; without it the next content write re-encrypts with the same `fileKey`. The failure is **silent** — no test will catch it unless a test specifically attempts to decrypt new content with the old `fileKey`. **How to avoid:** - -1. Run the new routing provider in read-parallel mode first: configure it alongside `delegated-ipfs.dev` and compare results for at least 48 hours (one full DHT record expiry cycle) before cutting writes over -2. Seed the new provider by having the TEE republisher publish to BOTH providers during the transition. The `publishSignedRecord()` method at line 200-204 sends to one delegated routing client; extend this to dual-write -3. Only cut over reads after the new provider consistently returns sequence numbers >= the DB cache for a representative sample of IPNS names -4. Keep DB cache as the authoritative fallback throughout -- this already works and should remain the primary resolution path +`rotateOne(N)` for a `kind: 'file'` node **must** mint a fresh `fileKey'` and set `contentRekeyPending` on the node record. The content write path must check `contentRekeyPending` and re-encrypt under the new `fileKey'`. Add a mandatory test: rotate a file node, publish a new version, assert a holder of the old `readKey`/`fileKey` cannot decrypt the new version (test item 2 in the design's Section 7.3). **Warning signs:** +`rotateOne` implementation has no branch on `node.kind === 'file'`. `contentRekeyPending` marker is never written. Content write path does not read or clear `contentRekeyPending`. Any test that checks "revoked reader cannot access new version" is absent from the suite. -- IPNS resolve latency spikes or increased null-from-network responses in Prometheus metrics (`cipherbox_ipns_resolves_total` by `source` label) -- 409 Conflict errors on publish that were not present before the switch -- Recovery tool fails to resolve IPNS records (it bypasses the API and hits the routing provider directly) - -**Phase to address:** IPNS Reliability phase (first phase of the milestone) +**Phase to address:** +Rotation engine phase (`packages/sdk-core` — `rotateReadFromNode`). Success criterion for the phase must require the CRIT-1 test passing before merge. --- -### Pitfall 2: Moving rootFolderKey to IPFS Creates a Hard Dependency on IPNS for Login +### Pitfall 3: M1 Generation Downgrade Defense Not Persisted = Silent Relay-Served Revocation Bypass **What goes wrong:** -Today, `rootFolderKey` is stored as ECIES-wrapped bytes in the `vaults` table (`encrypted_root_folder_key` column in `vault.entity.ts`). The login flow retrieves it via a direct PostgreSQL query -- fast, reliable, always available. The proposal to move it to an IPFS blob pointed at by the root vault IPNS record means that login now requires: (1) HKDF derivation of IPNS key, (2) IPNS resolution to get the CID, (3) IPFS fetch of the blob, (4) ECIES unwrapping. Steps 2 and 3 are network operations against infrastructure that has documented reliability issues. - -If IPNS resolution fails (which it does -- the entire motivation for this milestone), the user cannot log in because they cannot get their root folder key. The DB-cached CID fallback helps, but it means the database still stores CIDs pointing to encrypted root folder key blobs -- you have not actually eliminated the DB dependency, you have just moved what the DB stores from "ECIES-wrapped key bytes" to "CID of IPFS blob containing ECIES-wrapped key bytes." The net reduction in server-side state is zero unless you also eliminate the CID cache, at which point login depends entirely on IPNS reliability. +After a read-revoke rotation completes, a colluding or buggy relay can serve the revoked reader the pre-rotation (lower-generation) signed IPNS record indefinitely. The IPNS signature only covers the CID and sequence — not `generation`. The revoked reader's navigation succeeds from the old record. No in-memory client check survives a page refresh or app restart. The revocation appears to succeed but is undone by a relay rollback with zero signed evidence to the client. **Why it happens:** - -- The goal of "minimize database to auth-only" is laudable but creates a conflict with the reliability requirement -- The rootFolderKey is unique: unlike IPNS private keys which are HKDF-derivable, the root folder key is a random AES-256 key. It literally cannot be rederived. Losing access to it means losing the entire vault. -- Developers see the encryption key as "just data that should live on IPFS" without recognizing it is the single most critical piece of data in the entire system +No resolve path in the current codebase enforces a per-node `generation` check. `resolve_sequence_strict` tracks only `sequence` in-memory and loses it on restart. `VerifiedResolve` exposes `{cid, sequence_number}` and never decodes node metadata. The M1 defense is entirely new work, not an extension of existing checks. **How to avoid:** - -1. Keep `encrypted_root_folder_key` in the database as the primary source. Optionally mirror it to IPFS for recovery tool independence, but the DB copy is canonical for login. -2. If the goal is recovery-tool independence from the server, embed the wrapped key in the IPFS vault blob AND keep the DB copy. The recovery tool reads from IPFS; the normal login reads from the DB. Both paths work. -3. Never make the sole access path for root folder key depend on IPNS resolution. This is a "both, not either" situation. -4. Per `METADATA_EVOLUTION_PROTOCOL.md`, this is a breaking change to the root vault blob format. Version the blob format so old clients can still decrypt old-format blobs during migration. +Persist `{nodeId → highestGeneration}` durably (IndexedDB on web; sqlite in FUSE) seeded from the grant's `rootGeneration` as the owner-vouched floor. Thread this into `resolve_ipns_verified` (Rust) and `resolveIpnsRecord` (web) to fail closed on a generation regression. Add a server-side generation-forward gate in the publish path (`ipns.service.ts`) mirroring the existing sequence CAS. The durable map must survive process restart — any in-memory-only implementation silently removes the protection after restart. **Warning signs:** +`generation` check is implemented but stored in a React state variable or a non-durable JS variable. The sqlite/IndexedDB write for `highestGeneration` is absent. Publish gate in `ipns.service.ts` only gates `sequenceNumber`. Test 5 from the design (M1 generation downgrade) is absent or mocked. -- Login latency increases from <100ms to multi-second (IPNS resolve is median 11s on DHT per ProbeLab measurements) -- Login failure rate correlates with IPNS availability instead of being independent -- Recovery tool works but normal login fails (or vice versa) - -**Phase to address:** Database Minimization phase -- but this pitfall argues for NOT moving rootFolderKey off the DB at all, or at minimum maintaining a dual-write pattern +**Phase to address:** +API + resolve phase (atomic CAS, server-side generation gate); Web phase (durable M1 client map, IndexedDB); FUSE phase (durable sqlite map, `resolve_ipns_verified`). The server-side gate belongs with the API DB cutover phase; the client-side durable map belongs with web and FUSE implementation phases. --- -### Pitfall 3: Migrating Shares to IPFS Breaks Recipient-Initiated Discovery +### Pitfall 4: HIGH-4 — Add-During-Rotation Child Drop = Silent Data Loss **What goes wrong:** -The `shares` table enables recipient-initiated share discovery: a recipient calls `GET /shares/received` and the API queries `WHERE recipient_id = :userId`. If shares migrate to IPFS/IPNS (e.g., each user has an "inbox" IPNS record listing shares offered to them), the recipient must now know _where_ to look. In the current model, the server knows; in the IPFS model, either: +`rotateOne` re-seals the parent's read-body from an in-memory `children[]` list decoded at step 3. A concurrent upload that CAS-wins first adds a new `SealedChildRef` to that parent. When rotation retries from a 409, it re-seals the body from the **stale** in-memory child list — the new child is gone from the sealed body, and the parent republishes without it. The upload appeared to succeed (the upload itself returned 200) but the file is now invisible to all clients and unreachable without the CID, which no parent links to. + +**Why it happens:** +The 409 retry path re-seals from memory rather than re-fetching and re-decoding the parent node at the current sequence. It is a natural mistake to treat the CAS retry as "same operation, fresh sequence" rather than "refetch everything that could have changed." -(a) The sharer publishes to the recipient's inbox IPNS record -- but the sharer does not hold the recipient's IPNS private key and cannot sign the record, so this is architecturally impossible without a new key-sharing protocol. +**How to avoid:** +On every CAS-409 in `rotateOne`, **re-fetch the current parent node, re-decrypt the read-body, and merge any `SealedChildRef`s added since the initial decode** before re-sealing. The merge is: take the union of the pre-rotate child list and the freshly-fetched child list by `childId`, with the freshly-fetched entries winning for any conflict. Add the mandatory test (design Section 7.3, test 4): start a rotation, inject a concurrent upload via a separate client, assert the new child is present in the completed parent. -(b) The sharer publishes to their own IPNS record and the recipient polls all potential sharers -- but the recipient does not know who might share with them. +**Warning signs:** +`rotateOne` retry path re-seals from a captured variable rather than calling the decode path again. The `children` variable is captured in a closure that is not refreshed on retry. Test for concurrent modification is absent. -(c) A server-side index maps recipients to share IPNS records -- but this puts the discovery mechanism back in the database, negating the migration. +**Phase to address:** +Rotation engine phase (`packages/sdk-core` — `rotateReadFromNode`). The retry path must be reviewed explicitly in phase success criteria. -Additionally, the `share_keys` table stores per-file and per-subfolder ECIES-wrapped keys. Each key is specific to a recipient's public key. These cannot simply be "moved to IPFS" because they reference database UUIDs (`share_id`, `item_id`) for relational joins. The IPFS blob would need to embed all relationship data that currently lives in foreign keys and indices. +--- -**Why it happens:** +### Pitfall 5: HIGH-3 — Inner Grant Orphan = Grantee Permanently Locked Out + +**What goes wrong:** +A node deep in a subtree being rotated was independently shared to a third party (e.g., a single-file share to Carol). The rotation re-mints `readDescriptorRef` only for grants rooted **at the rotation root** — not for grants whose `rootNodeId` is a descendant. Carol's grant row now wraps a `readKey` that has been rotated away. Her `readDescriptorRef` is permanently invalid and there is no recovery path (the owner no longer has the old `readKey` to re-derive). Carol is locked out with no error message other than "decryption failed." -- The mental model "move everything to IPFS" does not account for relational operations like indexed queries on foreign keys -- Share discovery is fundamentally a server-side operation in a system where users cannot enumerate other users' IPNS records -- The existing share-revocation flow uses soft-delete with `revoked_at` timestamp and lazy key rotation -- this stateful lifecycle is hard to model in immutable IPFS blobs +**Why it happens:** +It is natural to implement grant re-mint as "re-mint all grants on the root node," missing that `shares.rootNodeId` can point to any node in the subtree, not just the root. Without an indexed scan of `shares` over the full rotated set, inner grants are silently skipped. **How to avoid:** - -1. Accept that shares, share_keys, and share_invites are server-side state that SHOULD stay in the database. They are access-control metadata, not user content. The server already sees `sharer_id`, `recipient_id`, and `ipns_name` in plaintext -- moving them to IPFS does not improve privacy. -2. If serverless share discovery is a goal (for BYO-IPFS or full decentralization), research CRDT-based IPNS inbox patterns BEFORE attempting migration. The todo at `2026-02-22-crdt-ipns-inbox-sharing.md` correctly flags this as research-only for this milestone. -3. Categorize DB tables into "can migrate" vs "must stay": - - **Can migrate:** `folder_ipns` (IPNS tracking), `pinned_cids` (quota tracking if BYO replaces it), vault keys (if dual-written) - - **Must stay:** `shares`, `share_keys`, `share_invites` (relational discovery), `device_approvals` (cross-device MFA), `users`, `auth_methods`, `refresh_tokens` (auth) +The rotation engine must query `shares WHERE rootNodeId IN (rotated_node_ids)` for each batch of nodes processed, re-mint `readDescriptorRef` for each non-revoked recipient, and bump `rootGeneration`. This query must be batched with the walk, not only run at the rotation root. Add the mandatory test (design Section 7.3, test 3): a single-file share exists at a leaf of the deleted subtree; assert the inner grantee's `readDescriptorRef` is re-minted, and the revoked recipient is cut. **Warning signs:** +Grant re-mint logic runs only at the initial call site in `rotateReadFromNode`, not in the per-node `rotateOne` callback. The `shares` table has no index on `rootNodeId`. No test exercises a grant rooted at an interior subtree node. -- Design documents propose "share IPNS records" without specifying how recipients discover them -- The words "poll all sharers" appear in a design doc (quadratic complexity) -- Share acceptance latency goes from instant (DB query) to seconds (IPNS resolution) - -**Phase to address:** Database Minimization phase -- but the actual prevention is to scope "minimize database" more narrowly than "move everything off" +**Phase to address:** +Rotation engine phase, with API support for the batched `shares` query. The API query path should be implemented and tested in the same phase as `rotateReadFromNode`, not deferred. --- -### Pitfall 4: BYO-IPFS Bypasses Server-Side Optimistic Concurrency +### Pitfall 6: Republisher Incrementing Sequence During Rotation = Stale-CID Rollback / Revocation Bypass **What goes wrong:** -CipherBox's conflict detection works because ALL IPNS publishes go through the API, which checks `expectedSequenceNumber` against the `folder_ipns` table before accepting a publish (lines 177-189 of `ipns.service.ts`). When a user configures a BYO-IPFS node and publishes IPNS records directly (client-to-node without API relay), the server never sees the publish and cannot check sequence numbers. Two devices could publish conflicting metadata for the same folder, and the DHT will keep whichever has the higher sequence number -- but neither device knows about the conflict. - -Worse: the TEE republishing service tracks sequence numbers in `ipns_republish_schedule`. If a BYO user publishes directly to their node, the TEE's sequence number becomes stale. The next TEE republish will create a record with a LOWER sequence number than what the user published, causing the DHT to reject it (or worse, a race where different DHT nodes hold different records). The user's IPNS records silently stop being republished. +The TEE 6-hour republisher currently does `+ 1n` on the sequence (`apps/tee-worker/src/routes/republish.ts:79`). During a rotation walk, a file node's pre-rotation record (with the old revoked-readable CID) gets republished at a **forward sequence** by the TEE. Because IPNS record selection is "higher sequence wins," the TEE's re-signed stale record now dominates the rotation's new record (which was published at an earlier sequence). The revoked reader receives the old, readable CID. The rotation appears to complete but is immediately undone by the next TEE cycle. **Why it happens:** - -- BYO-IPFS is designed for user sovereignty, but the entire concurrency model assumes server mediation -- The provider interface (`IpfsProvider`) only abstracts pin/unpin/fetch -- it does not abstract IPNS publishing, which is handled separately through `DelegatedRoutingClient` -- Client-direct upload to a user's Kubo node is the simplest implementation, but it bypasses every server-side check +The `sequence + 1` behavior was originally correct for availability (it ensures the refreshed record beats any cached older record on the network). The collision with rotation was not anticipated. The design's Section 4.6 initially incorrectly called the republisher "orthogonal" — this was corrected in the grilling session. **How to avoid:** - -1. Even with BYO-IPFS, require IPNS publishes to go through the API. The server does not need to contact the user's IPFS node for IPNS -- it just needs to see the publish request for concurrency checking and TEE enrollment. -2. Separate the concerns: BYO-IPFS is about WHERE data is pinned, not about HOW metadata is published. Pin encrypted blobs to the user's node; publish IPNS records through the API. -3. If client-direct IPNS publishing is a requirement (full decentralization), implement client-side conflict detection: before publishing, resolve the current IPNS record, compare sequence numbers, and abort on mismatch. This is weaker than server-side checks (no atomicity guarantee) but better than nothing. -4. Add a "last known sequence number" field to the BYO-IPFS configuration so the client can detect when the TEE's sequence diverges from the user's actual sequence. +The TEE republisher must **never increment the sequence**. The fix (design Section 6.2) is: the relay sends the marshaled existing signed record; the TEE parses it, verifies its signature, and re-emits a record with the **same value (CID) and same sequence** and only a later EOL. The IPNS equal-sequence/later-EOL tiebreak lets the refreshed record win network-wide without consuming a sequence. The publish path's `+ 1n` must be replaced on the TEE path. The atomic CAS gate (design Section 6.6) guards the idempotent renewal write identically. **Warning signs:** +`republish.ts` still contains `sequenceNumber + 1n` on the TEE path after the TEE contract rewrite phase. The `apps/tee-worker` phase is treated as a small change rather than a contract rewrite. No test validates "republisher does not increment sequence" (design test 12). -- BYO users report "stale" folder contents on other devices -- TEE republish success rate drops for BYO users specifically -- Sequence numbers in `ipns_republish_schedule` are lower than what IPNS resolves to - -**Phase to address:** BYO-IPFS phase -- design decision required before implementation begins +**Phase to address:** +TEE worker rewrite phase (`apps/tee-worker`). Must be completed before any rotation E2E testing, since a misconfigured republisher poisons rotation correctness. --- -### Pitfall 5: DHT Record Expiry During Routing Provider Migration +### Pitfall 7: Eager-vs-Lazy Scope-Exit Confusion = Massive Over-Rotation Storm on Private Deletes **What goes wrong:** -IPNS records on the DHT expire after 48 hours regardless of the validity field in the record. Kubo's default republish period is 4 hours, and CipherBox's TEE republishes every 6 hours. During a routing provider migration, if there is a gap where neither the old provider nor the new provider successfully publishes records, the 48-hour expiry clock keeps ticking. If the gap exceeds 48 hours (e.g., a weekend deployment where the new provider is misconfigured), ALL IPNS records for ALL users expire from the DHT simultaneously. Recovery requires republishing every single record -- but the TEE processes records in batches of 100 with a 6-hour cycle, so catching up on thousands of records takes many cycles. +The old `executeLazyRotation` rotated on every delete/move/rename. If the new implementation does not check "does this node have any covering grant?" before invoking `rotateReadFromNode`, every private (un-shared) delete triggers a full subtree rotation. A vault owner with 10,000 unshared files performs a routine folder delete and triggers 10,000 IPNS republishes, blocking the client for minutes and burning IPFS quota. This is not a security issue — it is a liveness / usability issue that will appear correct in unit tests but catastrophic at scale. + +**Why it happens:** +The rotation call site is added to delete/move/rename without the scope predicate. The predicate ("any active grant covers this node") requires querying `shares` by `rootNodeId` ancestry, which is non-trivial and easy to skip in early implementations. -The DB-cached CID fallback saves resolution, but only for users going through the API. The recovery tool (`recovery.html`) and any future BYO-IPFS clients resolving directly from the DHT will get nothing. +**How to avoid:** +The unified scope-exit rule is: **no covering grant → pure relink, zero rotations**. Implement `hasCoveringGrant(nodeId, ancestorIds)` as a required predicate at every delete/move/rename call site, gated against the relay-provided active grant set. Add a mandatory test (design Section 7.3, test 9): private delete with no grants → assert zero `rotateReadFromNode` invocations and zero `publishRecord` calls beyond the parent relink. + +**Warning signs:** +Delete/move/rename call sites invoke `rotateReadFromNode` unconditionally. `hasCoveringGrant` function does not exist. The scope predicate is a TODO comment. No test counts publish calls for an un-shared node deletion. + +**Phase to address:** +Rotation engine phase (scope-exit logic) and SDK + web mutation phase (delete/move/rename call sites). Both phases need the predicate; the predicate implementation belongs in the rotation engine phase. + +--- + +### Pitfall 8: Tombstone Enforcement Gap = Revoked Writer Keeps Publishing Forever + +**What goes wrong:** +Approach (c) write-revocation changes the Ed25519 keypair and k51 name. The old `ipns_records` row persists and the publish gate has zero tombstone awareness, so the revoked co-writer's cached Ed25519 key can publish to the old name indefinitely. The old name's signed record is still served by the relay's resolve path (it has a valid signature and a non-deleted row). Clients with bookmarks to the old name resolve stale (pre-rotation) content. The write revocation is cryptographically complete on the new name but operationally bypassed on the old one. **Why it happens:** +`unenrollIpns` only deletes the republish schedule row, not the `ipns_records` row, and adds no publish rejection flag. The publish gate in `ipns.service.ts` checks only sequence monotonicity and key-possession (`existing.publicKey.equals(...)`), not tombstone state. The tombstone is a new concept that requires a new DB column and a new gate check. + +**How to avoid:** +On write-revocation, mark the old row with `tombstonedAt` (design Section 5.5). The publish gate must check `tombstonedAt IS NOT NULL` and return 403/410 before the sequence check. Resolve must return 410 (or a tombstone marker in the response) for a tombstoned name, never stale content. The TEE republisher's renewal must also be blocked at the publish-gate tombstone check, so a malicious relay cannot feed the old signed record to an honest TEE and keep the revoked name's lease alive. Add test 20 (design Section 7.3): write to a tombstoned name → 403/410; resolve a tombstoned name → 410, not stale content. + +**Warning signs:** +`tombstonedAt` column does not exist in the migration. Publish gate checks only sequence and pubkey. Resolve falls through to the network record for tombstoned rows. `unenrollIpns` is used as the write-revocation teardown rather than a separate tombstone path. + +**Phase to address:** +API DB cutover and publish-gate phase. Tombstone schema and gate logic must be in the same migration as the write-revocation implementation. + +--- + +### Pitfall 9: Non-Atomic Publish CAS = Silent Lost Writes Under Concurrency -- The 48-hour DHT expiry is not widely known -- developers assume IPNS records persist until explicitly replaced -- The TEE republish cycle (6 hours) provides comfortable margin (48/6 = 8 missed cycles before expiry), creating false confidence -- Migration testing often happens on small test datasets where a full republish cycle completes quickly, masking the problem at scale +**What goes wrong:** +`publishRecord` is currently a non-atomic `findOne → gate → save` in TypeORM with no row lock and no conditional UPDATE. Two concurrent forward writers both at `dbSeq = N` both pass the gate check and the second `.save()` silently overwrites the first. The first writer receives `200` and believes the write landed. This is a data-loss bug that is invisible in single-client tests and appears only under realistic multi-device or rotation+upload concurrency. + +**Why it happens:** +TypeORM `.save()` is not atomic. The design's existing "sequence CAS" at `ipns.service.ts:301-317` is implemented as application-level logic, not a SQL `WHERE sequenceNumber = :expected` constraint. This gap existed in v1.1 and the design's Section 6.6 explicitly calls it out as unresolved. **How to avoid:** +Replace the `findOne → gate → save` with a single conditional UPDATE: -1. Never take the old routing provider offline until the new one has successfully processed at least one full republish cycle for ALL enrolled records -2. Monitor the `cipherbox_republish_schedule_total` gauge by status during migration -- any spike in `retrying` or `stale` status indicates the new provider is not accepting publishes -3. Before migration, calculate the worst-case catch-up time: `(total enrolled records / BATCH_SIZE) * republish_interval = (N / 100) * 6 hours`. For 1000 records, that is 60 hours to fully catch up from scratch -- already past the 48-hour expiry -4. Consider a one-time "emergency republish" endpoint that processes all records immediately (bypassing the 6-hour schedule) after a provider switch -5. Reduce BATCH_SIZE temporarily during migration to process more records per cycle, or run catch-up cycles more frequently +```sql +UPDATE ipns_records SET latestCid = :cid, sequenceNumber = :next, signedRecord = :rec, updatedAt = now() +WHERE ipnsName = :name AND sequenceNumber = :expected +``` -**Warning signs:** +Zero rows affected = 409 to the client. The EOL-only renewal (TEE lease) hits the same gate with `WHERE sequenceNumber = :loaded` so it can never regress `latestCid` from a stale in-memory row. Add test 16 (design Section 7.3): two concurrent publishes at the same `dbSeq` → exactly one 409, zero lost updates. -- `ipns_republish_schedule` entries accumulating in `retrying` or `stale` status -- IPNS resolution returning null for records that were previously resolvable -- Recovery tool stops working for all users simultaneously +**Warning signs:** +`publishRecord` still contains `findOne` followed by `save`. No `WHERE sequenceNumber = :expected` in any UPDATE. The "D-09 idempotent branch" short-circuits before the CAS rather than hitting the same conditional UPDATE. Test for concurrent writes is absent. -**Phase to address:** IPNS Reliability phase -- must be addressed in the migration runbook +**Phase to address:** +API DB cutover phase. This is a correctness fix that must land before any rotation E2E testing. --- -### Pitfall 6: Quota Tracking Becomes Unenforceable with BYO-IPFS +### Pitfall 10: TEE Relay-as-Signing-Oracle Trap = Write-Forgery Risk **What goes wrong:** -CipherBox tracks storage quota via the `pinned_cids` table: every upload goes through `IpfsController`, which calls `pinFile()` on the server-managed Kubo node and records the CID + size in `pinned_cids`. The user's total is summed against a 500 MiB limit. With BYO-IPFS, if uploads go directly to the user's node (client-direct), the server never sees the upload and cannot record the CID. The `pinned_cids` table becomes incomplete. Quota enforcement is either (a) impossible, (b) advisory only, or (c) requires the client to self-report, which is trivially forgeable. +If the TEE's lease-renewer contract is implemented as "receive a CID scalar from the relay, sign it with the wrapped IPNS key" rather than "receive a marshaled signed record, verify its signature, extend only the EOL," then the relay can coerce the enclave to sign any CID for any name it controls. A token-validation bug, SSRF, or auth bypass allows the relay to forge IPNS records under users' IPNS keys — exactly what a zero-knowledge system is designed to prevent. The enclave's security guarantee is reduced to "API server is trustworthy," negating the TEE's purpose. -The deeper issue: quota tracking exists to protect the CipherBox operator's IPFS node from unbounded storage consumption. If the user is pinning to their own node, the CipherBox operator has no storage cost and does not need to enforce quota. But the system currently conflates "enforce quota" with "track what the user has stored" -- and that tracking is also used for the recycle bin's CID unpinning (30-day retention + cleanup). +**Why it happens:** +The prior (stale) contract had the republisher sourcing `latestCid` from the `ipns_republish_schedule` snapshot. The simplest "fix" for the stale-CID bug is "refresh the snapshot from the canonical row" — which is still the scalar-signing trap. The structural fix (parse, verify, extend-EOL-only) requires more implementation complexity and a different enclave API shape. + +**How to avoid:** +The new enclave contract (design Section 6.4): (1) relay sends the **marshaled existing `signedRecord`** bytes, not a CID scalar; (2) the TEE parses it and **verifies the embedded signature**; (3) TEE re-emits a record with the **same value (CID) and same sequence**, only a later EOL. The enclave cannot be fed a CID it did not verify from a pre-existing valid record. Additionally, the three enclave bindings (design Section 6.7) must be present: internal epoch derivation (TEE derives `currentEpoch` from its own clock, never from relay-supplied scalars), name↔key binding (assert `publicKeyFromIpnsName(name) == pubkey(decryptedKey) == record.pubkey`), and migration durability (refuse to renew a key older than `currentEpoch - 1`). + +**Warning signs:** +TEE API receives a CID field from the relay rather than a marshaled signed record. TEE does not call a signature-verify function before re-emitting. Epoch scalars are passed in from the relay request body. Name↔key assertion is absent. + +**Phase to address:** +TEE worker rewrite phase (`apps/tee-worker`). The three Section 6.7 bindings must be in the same phase as the lease-renewer contract, not deferred. + +--- + +### Pitfall 11: `folder_ipns.public_key` Null-Row Footgun (Repeated Phase-60 Regression Pattern) + +**What goes wrong:** +The `folder_ipns.public_key` column is null for shared-folder rows and not always populated for other rows. Any code that reads `row.public_key` to obtain the Ed25519 pubkey for strict-verify will silently get `null` on shared-folder rows, causing either a null-dereference crash or a skipped signature verify (if the null check gates the verify call). This pattern caused two Phase-60 regressions in unit tests that were not caught because the tests did not use shared-folder rows. **Why it happens:** +The column exists in the schema as a convenience but is nullable and unreliable. Developers implementing new resolve or rotation paths naturally reach for `row.public_key` because it is on the same row — it is only null for a subset of rows that may not appear in unit test fixtures. + +**How to avoid:** +The design's Section 7.1 and 7.2 mandate: drop `folder_ipns.public_key` outright (it is derivable from the k51 name via `publicKeyFromIpnsName`). The migration must include `ALTER TABLE ipns_records DROP COLUMN public_key`. Every strict-verify call must recover the Ed25519 pubkey from the k51 name via `publicKeyFromIpnsName`, never from the column. Add a test with a shared-folder `ipns_records` row (null `public_key`) and assert that strict-verify works correctly. -- The existing quota model assumes all storage flows through the server -- Client-direct upload is the most obvious BYO architecture but breaks server-side bookkeeping -- Developers may add BYO pinning without considering that quota, recycle bin, and orphan cleanup all depend on the `pinned_cids` table +**Warning signs:** +Any code containing `row.public_key` or `ipnsRecord.publicKey` after the migration phase. `publicKeyFromIpnsName` is not imported in paths that previously used the column. Test fixtures only use owner rows (never shared-folder rows with null column). + +**Phase to address:** +API DB cutover phase (the migration drops the column). FUSE and SDK-core resolve phases must verify they use `publicKeyFromIpnsName` and never the dropped column. + +--- + +### Pitfall 12: winfsp Module Nesting Trap (`super::` vs `super::super::`) = Windows CI Failure, Invisible Locally + +**What goes wrong:** +The Rust `winfsp` feature cannot compile on macOS — `windows/*` modules (`#[cfg(winfsp)]`) are silently excluded from local `cargo check`/`cargo test`. A `super::` path inside a doubly-nested `pub mod implementation { pub mod write_ops { ... } }` that should be `super::super::` compiles cleanly on macOS and fails only on the Windows CI gate. This exact pattern (`super::content_fetch` instead of `super::super::content_fetch`) surfaced in Phase 55. The `node/v3` refactor deletes `spawn_file_meta_reencrypt` at `crates/fuse/src/metadata.rs:655` — its callers are `write_ops/implementation/rename.rs:248` **and** `platform/windows/write_ops.rs:1182` (the WinFsp twin). Missing the twin and getting the nesting wrong are two independent failure modes, both invisible on macOS. + +**Why it happens:** +macOS developers never see Windows-only modules compile. Path bugs in `platform/windows/write_ops.rs` look correct locally because the file is never loaded. Module nesting in `platform/windows/write_ops.rs` is one level deeper than the macOS equivalent, so a copy-paste from the macOS side uses the wrong number of `super::` prefixes. **How to avoid:** +Any FUSE phase touching `crates/fuse/src` must budget a Windows CI round-trip (`gh workflow run "Cargo Check & Test (Windows)"`) as a phase completion gate, not a merge afterthought. The phase plan must explicitly list `platform/windows/write_ops.rs` as a file to check alongside its macOS counterpart for every deletion or refactor. Use `grep -r "spawn_file_meta_reencrypt\|reencrypt" crates/fuse/` to locate all callers before beginning. -1. For BYO-IPFS users, skip server-side quota enforcement entirely. The user manages their own node's capacity. -2. Still require BYO uploads to be reported to the API (POST with CID + size after client-side pin succeeds). This maintains the `pinned_cids` ledger for features that depend on it (recycle bin, orphan detection) without requiring the server to handle the actual data. -3. Mark `pinned_cids` entries with a `provider` column ("server" | "byo") so the system knows which CIDs it can directly unpin vs. which require the client to unpin. -4. Accept that unpinning on a BYO node requires client cooperation -- the server cannot reach into the user's node. Document this limitation clearly. +**Warning signs:** +FUSE phase does not mention `platform/windows/write_ops.rs` in its blast radius. Phase success criteria do not include running the Windows Cargo CI gate. The `super::` depth in a new `platform/windows/` path is not independently reviewed. + +**Phase to address:** +FUSE implementation phase. Make the Windows CI gate a required check in the phase's success criteria, not optional. + +--- + +### Pitfall 13: Folderless Zustand/SDK folderTree Desync = "Folder not loaded" Class Under Rotation + +**What goes wrong:** +The scope-exit predicate and the rotation walk both require a consistent view of which IPNS nodes are covered by active grants (`folderTree` in the web app, `PublishCoordinator` / in-memory tree in FUSE). If `folderTree` in Zustand has not been reconciled to the current `sequenceNumber` before a delete or move triggers the scope predicate, the predicate may compute "no covering grant" for a node that actually has one (because the grant was added since the last sync) — silently skipping a required rotation. This is the same desync class that produced `#489` / `#494` ("Folder not loaded," "stale-sequence 409 + merge resurrecting deleted files"). + +**Why it happens:** +The web app's `folderTree` and the SDK client's `folderTree` are separate state stores. A direct SDK mutation (e.g., a background rotation resuming mid-session) can advance the SDK's tree without advancing Zustand. The scope-exit check at delete/move time reads the Zustand store, which is stale. + +**How to avoid:** +Reconcile `folderTree` against the current `sequenceNumber` before any scope-exit predicate evaluation (design Section 3.9). If reconciliation fails (cannot resolve the current IPNS state), the mutation **defers** rather than skips the rotation. The reconcile-before-publish discipline already exists in the codebase — apply it explicitly at the rotation entry point. Add a test that simulates a stale `folderTree` at the time a delete is invoked and asserts the rotation is either deferred or correctly computed from the refreshed tree. **Warning signs:** +Scope-exit predicate reads `useFolderStore.getState().folderTree` without first calling a reconcile. The rotation entry point in the web layer does not await a sync before computing coverage. No test uses a pre-stale Zustand store to trigger a delete. + +**Phase to address:** +Web implementation phase (the `executeLazyRotation` → `rotateReadFromNode` replacement). Add reconcile-before-rotate as a required step in the phase plan. + +--- + +### Pitfall 14: SDK-Core Coverage Excludes `index.ts` Barrels = Rotation Engine Silently Uncovered -- BYO users show 0 bytes used in quota display despite having files -- Recycle bin emptying fails silently for BYO-pinned CIDs (unpin calls go to the wrong node) -- Orphan CID detection reports all BYO CIDs as orphans +**What goes wrong:** +vitest coverage excludes `src/**/index.ts` barrels. If the `rotateReadFromNode` implementation or the `sealAesGcmAad` primitive is placed in a fat `index.ts` barrel file (to match existing sdk-core patterns), that code is excluded from coverage reports. A phase can hit the 80% coverage gate while leaving the most security-critical code unexercised by the coverage tool. The coverage gate passes, the phase completes, and silent gaps exist in the rotation engine. -**Phase to address:** BYO-IPFS phase +**Why it happens:** +The sdk-core pattern uses fat `index.ts` files. Developers naturally add new exports to the nearest `index.ts`. The coverage exclusion is a project-specific configuration that is easy to forget (it was discovered in Phase 55 when `folder/index.ts` split dropped coverage to 77.11%). + +**How to avoid:** +The rotation engine (`rotateReadFromNode`, `rotateOne`, `verifySubtreeClean`) and the job record persistence must be implemented in **named files** (e.g., `src/rotation/engine.ts`, `src/rotation/job.ts`), not in `index.ts` barrels. The design's Section 7.2 explicitly states: "in named files, not a fat `index.ts` barrel — coverage excludes barrels." The phase plan must name the output files, not just the API surface. + +**Warning signs:** +`rotateReadFromNode` is exported from `src/index.ts`. Phase plan says "add to sdk-core" without specifying file names. Coverage report shows high percentage but `src/index.ts` is in the exclusion list. + +**Phase to address:** +`packages/sdk-core` rotation engine phase. File naming must be specified in the phase plan as a success criterion. --- -## Technical Debt Patterns +### Pitfall 15: Web vitest Only Runs `*.test.ts` Files = Rotation Specs Silently Skipped + +**What goes wrong:** +`apps/web` vitest configuration's `include` pattern is `src/**/*.test.ts`. Any test file named `*.spec.ts` is silently skipped and never appears in CI results. If rotation-related tests for the web layer are created with `.spec.ts` naming (which is common in NestJS-pattern tests and SDK integration tests), they never run. The CI passes and the tests are never executed. -Shortcuts that seem reasonable but create long-term problems. +**Why it happens:** +The web app adopted the `*.test.ts` convention but it is not enforced by any lint rule. Test file naming is easy to get wrong when copying from SDK or API test patterns that use `.spec.ts`. + +**How to avoid:** +All test files for `apps/web` must use `.test.ts` naming. When creating new tests in the web phase, verify file naming convention before committing. A grep across the web test directory for `.spec.ts` is a cheap phase entry check. + +**Warning signs:** +New test files in `apps/web/src/` with `.spec.ts` extension. CI shows no new test cases despite new test files being committed. -| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable | -| -------------------------------------------------------------------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Dual-write to old and new routing provider "forever" | Safe migration | Double the publish latency, double the failure surface, code complexity of maintaining two routing clients | Only during migration transition; set a hard deadline to remove the old provider within 2 weeks of cutover | -| Skip IPNS signature verification on DB-cached resolutions | Faster resolve, simpler code | DB-cached CIDs are not cryptographically verified -- a compromised DB could serve malicious CIDs | Acceptable only if the DB is trusted infrastructure. The web client at `ipns.service.ts:160-169` already verifies signatures from network resolves but skips it for DB-fallback results | -| BYO users self-report CID sizes for quota | Simple client-direct architecture | Users can lie about sizes; quota is advisory only | Acceptable for tech demonstrator. For production, require server verification of at least one byte-range to confirm size | -| Use DB-cached CID as primary resolution (skip IPNS network entirely) | Instant resolve, no DHT latency | DB becomes single point of truth for metadata location; IPNS becomes decorative | Acceptable as interim step. The current code already prefers DB when it has a newer sequence number. Making this explicit reduces complexity | -| Hardcode Someguy URL instead of making it configurable | Faster initial implementation | Cannot switch providers without code change and redeploy | Never -- use `DELEGATED_ROUTING_URL` env var (already exists in `delegated-routing.client.ts:22-25`). Zero cost to keep it configurable | -| Measure performance baselines during staging only | No production impact | Staging has different latency, load, and network characteristics than production | Never for baselines intended to represent production. Staging baselines are useful only for relative comparison (before/after a change) | +**Phase to address:** +Web implementation phase. Add a check in the phase plan: `find apps/web/src -name "*.spec.ts"` must return empty. --- -## Integration Gotchas +### Pitfall 16: Zeroization of Caller-Owned Reused Buffers in SDK E2E = 400 "publicKey does not correspond" -Common mistakes when connecting to external services or changing existing integrations. +**What goes wrong:** +If `rotateReadFromNode` or any rotation helper zero-clears a key buffer that was passed in by the caller (rather than a buffer it owns), subsequent SDK operations that reuse that caller-owned buffer (e.g., the user's `publicKey` or a grant's `readKey`) will operate on a zeroed buffer and produce 400 errors from the API ("publicKey does not correspond"). This broke 48/89 SDK E2E tests in a prior incident. The failure is a runtime regression at SDK E2E level — unit tests will not catch it because they do not share buffers across calls. -| Integration | Common Mistake | Correct Approach | -| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Someguy (self-hosted delegated routing) | Deploying Someguy without DHT bootstrap peers, resulting in an isolated node that cannot resolve any existing IPNS records | Configure Someguy with the Amino DHT bootstrap peers. Run it alongside Kubo so it benefits from Kubo's established DHT connections. Verify resolution of known IPNS names before cutting over | -| Kubo DHT direct (Routing.Type=auto) | Assuming the existing Kubo node already participates in DHT for IPNS. CipherBox's Kubo may be configured with DHT disabled (using only delegated routing) | Check `Routing.Type` in Kubo config. If it is "none" or "custom" with only delegated HTTP routers, IPNS DHT publishing/resolution is not happening through Kubo itself | -| IPFS Pinning Service API (for BYO) | Treating the Pinning Service API as equivalent to Kubo's `/api/v0/add`. The Pinning Service API is async -- `POST /pins` returns a `requestid` and pinning happens in the background | Poll pin status before assuming content is available. The CID may not be retrievable from the BYO node until pinning completes. Upload UX must reflect this ("pinning..." not "uploaded") | -| Recovery tool (`recovery.html`) | Changing the API's routing provider but forgetting the recovery tool resolves IPNS directly at line 363. The recovery tool is a standalone HTML file with no build system -- it does not use the API client | Update the recovery tool's default gateway URL. Better: make the recovery tool configurable to use the same routing provider as the API, or provide a recovery-specific resolve endpoint | -| TEE republishing during provider switch | The TEE worker calls the API's `publishSignedRecord()` method, which calls `DelegatedRoutingClient.publish()`. If the URL changes, the TEE's publishes go to the new provider but existing DHT records on the old provider expire | During transition, have the republisher publish to both providers. The TEE itself does not need to change -- only the API's publish path needs to dual-write | -| Prometheus metrics endpoint | Adding IPNS latency histograms that create high-cardinality labels (e.g., per-ipnsName timings) | Use fixed label values (e.g., `operation=publish/resolve`, `source=network/db_cache`). The existing `httpRequestDuration` histogram correctly normalizes routes. Follow the same pattern for new IPNS-specific histograms | +**Why it happens:** +Zeroization discipline requires clearing key material after use. A callee that receives a buffer it does not own (e.g., the caller's `readKey`) zeros it "for safety" on completion, not realizing the caller reuses it across multiple operations. The bug is invisible in isolated unit tests. + +**How to avoid:** +Rotation helpers must zero only key material they allocate. A caller-passed `readKey` buffer must be treated as owned by the caller; if the helper needs to zero something, it must zero its own derived copy. Document this in the function's JSDoc. The SDK E2E suite (`tests/sdk-e2e`) must run after any rotation helper is added — it is the only test that exercises real client→API IPNS publish/resolve round-trips with shared buffers. + +**Warning signs:** +Any rotation helper calling `key.fill(0)` on a parameter buffer. SDK E2E failures of the form "400 publicKey does not correspond" after a rotation call. Unit tests all pass while SDK E2E shows `48/89` failures. + +**Phase to address:** +`packages/sdk-core` rotation engine phase. Add a reminder in the phase plan: zero only locally-allocated buffers, never caller parameters. SDK E2E must pass before phase sign-off. --- -## Performance Traps +### Pitfall 17: SDK E2E Is the Only Real Publish/Resolve Gate — Desktop E2E Is Dispatch-Gated + +**What goes wrong:** +`tests/sdk-e2e` is the only test suite that exercises a real client→API IPNS publish/resolve round-trip. It is not run on PRs by default (requires a live API). The desktop E2E (`apps/desktop`) is dispatch-gated — `CI E2E Tests` skips desktop on main pushes that do not touch desktop paths. A rotation engine merged to `packages/sdk-core` and a FUSE rewrite merged to `crates/fuse` can both pass unit CI and land on `main` without the full integration ever running. + +**Why it happens:** +The gating is intentional (cost/speed), but it creates a coverage gap for the rotation feature that is more significant than for stable features — rotation is the first feature that exercises the full publish-rotate-resolve-verify loop at real IPNS latency. + +**How to avoid:** +After the rotation engine phase and after the FUSE phase, run the SDK E2E suite manually (`tests/sdk-e2e` with the local API stack up) and run the desktop E2E via `gh workflow run "CI E2E Tests" --ref `. Do not merge the rotation engine or FUSE phases without at least one SDK E2E pass and one Windows CI pass. These must be explicit gates in the phase completion checklist, not post-merge follow-ups. -Patterns that work at small scale but fail as usage grows. +**Warning signs:** +Phase sign-off does not include "SDK E2E run locally." Desktop E2E CI gate is not triggered on the FUSE phase branch. Phase completion criteria only reference unit tests. -| Trap | Symptoms | Prevention | When It Breaks | -| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Full DHT resolution on every IPNS resolve | Login takes 11+ seconds (DHT median); folder navigation feels frozen | Use DB-cached CID as primary resolve path; DHT resolve as background refresh (the current code approximates this but still blocks on DHT first) | Immediately -- median 11s latency per ProbeLab measurements on Amino DHT. Already broken for UX | -| TEE republishing all records in sequence | Republish cycle exceeds the 6-hour interval for users with many folders/files; records start expiring before the next cycle | The current BATCH_SIZE=100 with concurrency helps. Monitor `republish_entries_processed_total` vs cycle duration. Add per-user prioritization | At ~600+ enrolled records per cycle (100 batches x 6s avg per batch = 600s = 10 min total, still fine; but network issues or TEE latency can push individual batches to 60s+, at which point 100 batches x 60s = 100 min, approaching limits) | -| Instrumenting every IPFS operation with synchronous logging | Console.log/warn calls on hot paths (the codebase has 50+ console calls per CONCERNS.md) block the event loop; adding structured logging to IPFS pin/unpin adds latency to user-facing uploads | Use async log shipping (not synchronous console). Instrument at the histogram level (timing), not the log level (string formatting). The existing Prometheus setup is the right model | Immediately -- any synchronous logging on IPFS hot paths adds measurable latency | -| Client-side performance measurement using `Date.now()` | Sub-millisecond operations appear as 0ms; timer coarsening in browsers hides real variance | Use `performance.now()` for client-side timing, `process.hrtime.bigint()` for server-side (already used in `http-metrics.interceptor.ts:13`). Set histogram buckets appropriately for the expected range | Immediately -- `Date.now()` has millisecond resolution which is insufficient for sub-ms operations | -| Baseline measurements taken during development with hot caches | Baselines show artificially good numbers because Kubo's blockstore cache, OS page cache, and browser cache are warm | Always include cold-start measurements: restart Kubo, clear browser cache, flush OS page cache. Report both cold and warm baselines | When baselines are used to set SLO thresholds -- warm-cache numbers are artificially optimistic | -| BYO-IPFS with user's home node behind NAT | Pin succeeds locally but content is not retrievable by other peers because the node is not reachable. User reports "upload succeeded but file is missing" | Verify content availability after pin by attempting retrieval from a different peer. Warn users about NAT traversal requirements. Kubo's relay/AutoNAT helps but is not guaranteed | Immediately for any user behind consumer NAT (most home connections) | +**Phase to address:** +Rotation engine phase (sdk-core) and FUSE phase — both must add SDK E2E and desktop E2E as explicit phase gates. --- -## Security Mistakes +### Pitfall 18: `parseCachedRecord`-Null Fall-Through Serves Ungated Network Records + +**What goes wrong:** +When `parseCachedRecord` returns null for a shared-folder `ipns_records` row (which legitimately has a null `signedRecord`), the resolve path currently falls through to the network record without applying any sequence floor. A malicious relay can exploit this by serving an old, legitimately-signed, low-sequence network record to a shared-folder reader without triggering any gate. This is not the same as the tombstone case; it is the normal shared-folder resolve path being ungated. + +**Why it happens:** +The null check was originally added to handle corruption or absence. The shared-folder case (where `signedRecord` is legitimately null because the owner holds the key and the shared-folder row is a skeleton) was not distinguished from the corruption case. The design's Section 6.5 makes the case-split explicit: null-signedRecord for shared-folder rows → apply `seq ≥ storedSeq` floor from the DB `sequenceNumber` column; `signedRecord`-CID mismatch → fail closed. + +**How to avoid:** +Implement the two-case split explicitly in `resolveRecord`: (1) if `signedRecord` is null and the row's `sequenceNumber` is set, apply `seq ≥ storedSeq` floor to the network record; (2) if `signedRecord` is present but its decoded CID disagrees with `latestCid`, fail closed with a 500/503, never falling through. Add test 15 (design Section 7.3): verify both cases behave correctly. -Domain-specific security issues beyond general web security, relevant to the IPFS infrastructure changes. +**Warning signs:** +The null check in `resolveRecord` is a single `if (signedRecord == null) return networkResult` with no floor. The two-case distinction is absent. No test uses a legitimate null-`signedRecord` shared-folder row. -| Mistake | Risk | Prevention | -| ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Exposing Kubo API (port 5001) to BYO users for "direct pinning" | Kubo API has no authentication. Any client with network access can pin/unpin/add content, exhaust disk, or read any pinned content | Never expose Kubo API directly. Use the IPFS Pinning Service API (port-separated, auth-capable) or proxy through the CipherBox API with JWT auth | -| Storing BYO-IPFS node credentials (API tokens, auth headers) in the database without encryption | Server compromise exposes all BYO users' IPFS node credentials | Encrypt BYO config with the user's publicKey (ECIES). Store only the wrapped blob. The client unwraps and uses the credentials client-side | -| Self-hosted Someguy without rate limiting | DoS vector: anyone can flood the delegated routing endpoint with publish/resolve requests, exhausting DHT connections | Deploy Someguy behind a reverse proxy with rate limiting. The existing `delegated-routing.client.ts` handles 429 responses (lines 62-69), so rate-limited responses are gracefully retried | -| Performance baselines that include auth tokens in timing data | Timing side-channels could reveal information about token validation (e.g., shorter response for invalid tokens vs. valid tokens with expired sessions) | Exclude auth endpoints from public-facing performance dashboards. Report only aggregate histograms, not per-request timings. The existing Prometheus setup correctly does this | -| IPNS records signed with leaked TEE epoch keys | If a previous-epoch TEE private key is compromised, an attacker could forge IPNS records for any user whose key was encrypted with that epoch | The 4-week grace period for epoch rotation means old keys are eventually discarded. Ensure the new routing provider validates IPNS record signatures (Someguy does this inherently via the IPNS spec). Monitor for unexpected sequence number regressions | +**Phase to address:** +API resolve hardening phase (same as the atomic CAS and the generation gate). --- -## UX Pitfalls +### Pitfall 19: `spawn_file_meta_reencrypt` — Two Callers, Not One + +**What goes wrong:** +The `node/v3` content self-seal (Section 2.9) makes file moves pure re-links (no re-encryption needed). This kills `spawn_file_meta_reencrypt` (`crates/fuse/src/metadata.rs:655`). The function has two callers: `write_ops/implementation/rename.rs:248` (visible on macOS) and `platform/windows/write_ops.rs:1182` (the WinFsp twin, invisible on macOS). If only the first caller is deleted, `platform/windows/write_ops.rs` retains a call to a deleted function and fails on the Windows CI gate with a compilation error. + +**Why it happens:** +The two-caller structure mirrors the macOS/Windows split that exists throughout `crates/fuse`. The Windows twin is only visible under `#[cfg(winfsp)]` and requires a CI round-trip to verify. This exact pattern (a function called from both paths) caused the Phase-55 `super::` nesting bug. -Common user experience mistakes when adding these infrastructure features. +**How to avoid:** +Before beginning the FUSE phase, run `grep -r "spawn_file_meta_reencrypt\|reencrypt" crates/fuse/` and list all callers in the phase plan. Treat the deletion as complete only after both callers are removed and the Windows Cargo CI gate passes. + +**Warning signs:** +FUSE phase plan only mentions `rename.rs` as the caller to update. `platform/windows/write_ops.rs` is not in the blast radius list. -| Pitfall | User Impact | Better Approach | -| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Showing IPNS resolution latency as a loading spinner without explanation | Users see a 3-11 second spinner after every folder navigation and assume the app is broken | Show folder contents from DB cache immediately, then refresh in background when IPNS resolves. The current 30-second polling partially achieves this, but initial navigation still blocks on resolve | -| BYO-IPFS setup requires knowing Kubo API URL, authentication, and IPNS config | Only technically sophisticated users can configure it; everyone else is stuck with default | Provide auto-detection: if user enters a hostname, probe standard ports (5001 for Kubo API, 9097 for Pinning Service API). Offer presets for common providers (web3.storage, Filebase, Pinata) | -| Performance baseline results displayed as raw numbers | Users see "IPNS resolve: 11234ms" and panic, not understanding this is a DHT operation | Present baselines as traffic-light indicators (green/yellow/red) with context: "IPNS resolve: 11.2s (typical for DHT; cached resolves are <100ms)" | -| Silent fallback from IPNS to DB cache with no indication | User thinks they have "real IPFS" but resolution always falls back to the database. False sense of decentralization | Show a subtle indicator when resolution source is DB-cache vs. IPNS network. Log it in a diagnostics panel for advanced users | -| BYO-IPFS node goes offline and user loses access to files | User pinned files to their home node, it crashes, content is lost because no other node has copies | Warn users that BYO means single-point-of-failure unless they also pin to a backup. Offer "pin to both server and BYO" as default mode | +**Phase to address:** +FUSE implementation phase. This is a two-file deletion; both files must be listed in the phase plan explicitly. --- -## "Looks Done But Isn't" Checklist +### Pitfall 20: Cold-Node Filename Leak via Plaintext `SealedChildRef.name` on Un-Rotated Tail + +**What goes wrong:** +`SealedChildRef.name` is plaintext **within** the parent's sealed read-body. A revoked reader who already decrypted the parent's read-body (before the rotation reached the parent) has a copy of all child names, including names of items added **after** the revocation but before the rotation reached that node. The lazy-walk variant of the old `executeLazyRotation` had this bug (MED-5 in the design). The eager walk of `rotateReadFromNode` bounds the window, but does not eliminate it: filenames added to a not-yet-rotated node are visible to the revoked reader. + +**Why it happens:** +Name confidentiality within the sealed body depends on the parent node being rotated before the revoked reader can re-read the parent. During a multi-hour rotation walk over a large subtree, names added to interior nodes between the root step and the node's own rotation step are exposed. The root step cuts navigation from the entry point, but a reader who pre-fetched the interior node's CID can still unseal the body with the cached `readKey`. -Things that appear complete but are missing critical pieces. +**How to avoid:** +This is an accepted bounded residual under the eager-walk model (ADR 0002): the exposure window is ≤ the walk duration and the root step cuts the entry point immediately. Do NOT build a lazy-walk variant that leaves this window indefinitely open. The phase plan must document this residual in the rotation engine phase's security properties section so it is explicit, not an oversight. The optional per-file "re-encrypt now" path (design Section 4.1) is the mitigation for high-sensitivity cases. + +**Warning signs:** +A deferred implementation adds lazy-walk logic as an "optimization" without documenting the MED-5 filename-leak regression. Phase success criteria do not state the eager-walk invariant. -- [ ] **Routing provider replacement:** New provider resolves IPNS names -- but verify it also PUBLISHES reliably. Resolution can fall back to DB; publishing cannot. Test publish + resolve round-trip, not just resolve. -- [ ] **DB-to-IPFS migration for folder_ipns:** folder_ipns rows exist in IPFS blobs -- but verify the TEE republish service still works. It reads `encrypted_ipns_key` and `sequence_number` from the schedule table, not from IPFS. These columns cannot be eliminated if TEE republishing continues. -- [ ] **BYO-IPFS pin integration:** Files pin to user's node -- but verify they are retrievable by the CipherBox API for serving to other devices. Content must be discoverable on the IPFS network, not just locally pinned. -- [ ] **Performance baselines recorded:** Histograms populated in Prometheus -- but verify baselines include error cases. A P99 that excludes 502/504 errors paints a falsely rosy picture. -- [ ] **Recovery tool updated for new routing:** recovery.html points to new routing endpoint -- but verify it works WITHOUT the CipherBox API running. The entire point of the recovery tool is server-independent vault access. -- [ ] **Orphaned IPNS cleanup:** IPNS records migrated off DB -- but verify the orphaned IPNS records from pre-migration are cleaned up. The CONCERNS.md already flags orphaned IPNS records as tech debt. -- [ ] **Sequence number continuity:** After migration, first publish from client succeeds -- but verify the sequence number is correctly incremented from the pre-migration value, not reset to 1. A reset would cause the DHT to prefer the old (higher-sequence) record. -- [ ] **BYO quota display:** Settings page shows BYO config -- but verify the vault storage usage display updates correctly. If `pinned_cids` is incomplete, the UI shows wrong usage. +**Phase to address:** +Rotation engine phase. The invariant "eager walk only; lazy walk is deferred and documented as introducing MED-5 regression" must be stated in the phase plan. --- -## Recovery Strategies +## Technical Debt Patterns -When pitfalls occur despite prevention, how to recover. +| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable | +| -------- | ----------------- | -------------- | --------------- | +| Skipping KAT in crypto primitive phase | Faster initial PR | Silent total decryption failure at first cross-language read | Never | +| In-memory generation high-water (not persisted) | Simpler implementation | M1 defense evaporates on page reload / app restart | Never | +| Mocking IPNS in rotation crash-safety tests | Faster test execution | Resume/idempotency never exercises real CAS race | Never for crash-safety suite | +| Adding `rotateReadFromNode` to `index.ts` barrel | Consistent with existing sdk-core pattern | Implementation excluded from coverage; 80% gate passes with uncovered rotation engine | Never | +| Using `folder_ipns.public_key` column instead of `publicKeyFromIpnsName` | Fewer function calls | Null-row crash on shared-folder rows (known Phase-60 pattern) | Never | +| Running only unit tests to sign off FUSE phase | Fast phase completion | Windows CI winfsp path has undetected `super::` nesting bugs | Never | +| Treating TEE tombstone and schedule-row deletion as equivalent | Simpler teardown | Revoked name keeps being renewed; publish gate is never closed | Never | -| Pitfall | Recovery Cost | Recovery Steps | -| --------------------------------------------------------------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Sequence number divergence after provider switch | LOW | DB is authoritative. Force-republish all records from DB state to new provider. Client-side sequence numbers in Zustand stores may be stale -- user must refresh (F5). | -| rootFolderKey inaccessible (IPNS resolution failure) | CRITICAL if DB copy removed; LOW if DB copy retained | If DB copy exists: serve from DB (already the fallback). If DB copy was removed: user must use recovery tool with their private key to re-derive IPNS key, resolve from DHT (may be expired), and fetch from IPFS. If both fail: vault is permanently inaccessible. | -| Shares unreachable after migration to IPFS | HIGH | Must revert to DB-backed shares. If DB tables were dropped: reconstruct from IPFS blobs by scanning each user's share IPNS records, but recipient discovery requires the server index to be rebuilt from scratch. | -| BYO concurrency conflict (two devices wrote different metadata) | MEDIUM | Detect by comparing IPNS records from multiple DHT lookups. The record with the higher sequence number wins. The losing device must fetch the winning metadata and re-apply its changes on top (manual merge). No automated merge exists in the current architecture. | -| 48-hour DHT expiry (all records lost from DHT) | MEDIUM | DB-cached CIDs still work for API-mediated resolution. To restore DHT records: trigger emergency republish of all entries. At BATCH_SIZE=100, 1000 records = 10 batches. If each batch takes 30s, full recovery = 5 minutes for TEE signing + publishing time. | -| Quota tracking broken for BYO users | LOW | Run a reconciliation job: for each BYO user, fetch their folder metadata from IPNS, enumerate all CIDs, check pin status on their node, update `pinned_cids` table. Can be run as a one-time migration or periodic background job. | -| False performance baselines from observer effect | LOW | Re-run baselines with instrumentation disabled, compare. If overhead > 5%, switch to sampling-based measurement (10% sample rate). Document the measurement methodology alongside the baseline numbers. | +--- + +## Integration Gotchas + +| Integration | Common Mistake | Correct Approach | +| ----------- | -------------- | ---------------- | +| `pnpm api:generate` | Skipping after `apps/api` DTO changes for `shares`, `ipns_records` | Always run after API changes; `check-api-client.sh` pre-commit hook enforces this, but only when staged correctly | +| SDK E2E (`tests/sdk-e2e`) | Running against a mocked API or skipping entirely | Must run against a live local stack (`docker compose up` + `pnpm --filter @cipherbox/api dev`); it is the only real publish/resolve gate | +| Desktop E2E | Checking CI result on a main push that didn't touch `apps/desktop` | Trigger explicitly: `gh workflow run "CI E2E Tests" --ref ` | +| TypeORM migrations | Using `synchronize: true` locally to verify DB shape | `synchronize` is off everywhere; write explicit migration and verify it with `migrationsRun: true` | +| Atomic CAS | Using TypeORM `.save()` for sequence-gated publishes | Use raw `UPDATE … WHERE sequenceNumber = :expected` and check rows-affected | +| TEE epoch scalars | Reading epoch from relay request body in the enclave | TEE derives `currentEpoch` from its own clock + schedule; relay-supplied epoch scalars are untrusted and ignored | --- -## Pitfall-to-Phase Mapping +## "Looks Done But Isn't" Checklist -How roadmap phases should address these pitfalls. +- [ ] **Crypto primitive:** KAT fixture committed and asserted by both `packages/crypto/__tests__` and a Rust `#[test]` — verify all four role bytes are in the fixture +- [ ] **Rotation engine:** `rotateOne` for file nodes mints `fileKey'` and sets `contentRekeyPending` — verify by checking for `node.kind === 'file'` branch in the implementation +- [ ] **Rotation engine:** 409-retry path re-fetches and re-merges the parent's `SealedChildRef` list — verify no captured stale `children` variable +- [ ] **Rotation engine:** Scope-exit predicate exists and gates every delete/move/rename call site — verify with a test counting zero publishes for an un-shared delete +- [ ] **Grant re-mint:** Inner grants (those with `rootNodeId` pointing to a node inside a rotated subtree, not the root) are re-minted — verify test 3 from the design test strategy +- [ ] **Tombstone:** `tombstonedAt` column exists; publish gate checks it before the sequence gate; resolve returns 410; TEE renewal is blocked at the same gate +- [ ] **Republisher:** `apps/tee-worker` no longer increments `sequenceNumber`; same sequence is re-signed with extended EOL only +- [ ] **M1 generation high-water:** Persisted to IndexedDB (web) / sqlite (FUSE), survives restart, seeded from `rootGeneration` on first grant receipt +- [ ] **Atomic CAS:** `publishRecord` uses a conditional UPDATE; `.save()` is gone from the publish path +- [ ] **`folder_ipns.public_key` column:** Dropped in migration; no code reads `row.public_key` or `record.publicKey` from the DB row +- [ ] **`platform/windows/write_ops.rs`:** Updated alongside the macOS FUSE path for every file touched in the FUSE phase +- [ ] **Windows CI gate:** `Cargo Check & Test (Windows)` passes for every FUSE phase PR +- [ ] **SDK E2E:** Runs and passes after rotation engine and FUSE phases before merge +- [ ] **api:generate:** Regenerated client committed alongside every API endpoint/DTO change +- [ ] **`apps/web` tests:** All new test files use `.test.ts` not `.spec.ts` +- [ ] **sdk-core rotation files:** Named files (not `index.ts`); coverage report shows them as covered + +--- + +## Pitfall-to-Phase Mapping -| Pitfall | Prevention Phase | Verification | -| ------------------------------------------ | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| P1: Sequence number divergence | IPNS Reliability | Run dual-provider for 48+ hours; compare sequence numbers across providers for 100% of IPNS names | -| P2: rootFolderKey on IPNS | Database Minimization | Verify login works when IPNS is completely down (kill Someguy, disable DHT); login must still succeed via DB | -| P3: Share discovery on IPFS | Database Minimization | N/A -- prevention is to NOT migrate shares. Verify shares still work after other tables are migrated | -| P4: BYO bypasses concurrency | BYO-IPFS | Simulate two BYO devices publishing simultaneously; verify conflict is detected and one publish is rejected or flagged | -| P5: DHT record expiry during migration | IPNS Reliability | Before cutover, verify new provider has successfully republished all records at least once. Check that zero `stale` entries exist in `ipns_republish_schedule` | -| P6: Quota tracking with BYO | BYO-IPFS | Upload 10 files via BYO, verify `pinned_cids` table has 10 entries with correct sizes. Empty recycle bin, verify CIDs are unpinned on user's node | -| P7: Instrumentation overhead | Performance Baselines | Run baseline suite with and without instrumentation. Verify overhead < 5% on P95 latency. If higher, reduce sampling rate | -| P8: Recovery tool broken by routing change | IPNS Reliability | Run full recovery flow using ONLY recovery.html (no API) with the new routing provider. Verify all files are recoverable | +| Pitfall | Prevention Phase | Verification | +| ------- | ---------------- | ------------ | +| AAD byte-encoding drift (TS/Rust KAT) | Crypto primitive phase (Phase 1) | KAT fixture committed and passing in both `packages/crypto` and `crates/crypto` | +| CRIT-1 content-key rotation omitted | Rotation engine phase (sdk-core) | Test: old `fileKey` cannot decrypt new version after rotation | +| M1 generation downgrade not persisted | API (server gate) + Web (IndexedDB) + FUSE (sqlite) phases | Test: generation regression rejected after restart | +| HIGH-4 add-during-rotation child drop | Rotation engine phase | Test: concurrent upload is present in rotated parent | +| HIGH-3 inner grant orphan | Rotation engine phase + API (query support) | Test: inner grantee's `readDescriptorRef` re-minted on subtree rotation | +| Republisher sequence increment | TEE worker rewrite phase | Test: republish does not increment sequence; stale CID not re-signed | +| Over-rotation on private deletes | Rotation engine + web/SDK mutation phases | Test: zero publish calls for un-shared delete | +| Tombstone enforcement gap | API DB cutover + publish-gate phase | Test: write to tombstoned name → 403/410; resolve → 410 | +| Non-atomic publish CAS | API DB cutover phase | Test: concurrent publishes → exactly one 409 | +| TEE relay-as-signing-oracle | TEE worker rewrite phase | Test: TEE epoch self-derives; name↔key binding asserted | +| `folder_ipns.public_key` null-row footgun | API DB cutover phase (migration drops column) | No code references `row.public_key`; shared-folder strict-verify test | +| winfsp nesting trap | FUSE implementation phase | `Cargo Check & Test (Windows)` CI gate required in phase success criteria | +| folderTree/SDK desync | Web implementation phase | Reconcile-before-rotate at every scope-exit entry point | +| sdk-core barrel coverage gap | sdk-core rotation engine phase | Named files; coverage report includes rotation engine files | +| `*.spec.ts` silently skipped in web vitest | Web implementation phase | `find apps/web/src -name "*.spec.ts"` returns empty | +| Zeroization of caller-owned buffers | sdk-core rotation engine phase | SDK E2E passes after rotation helpers added | +| SDK E2E / desktop E2E dispatch-gating | Rotation engine phase + FUSE phase | SDK E2E run manually; desktop E2E triggered explicitly before merge | +| `parseCachedRecord`-null fall-through | API resolve hardening phase | Test: null-signedRecord shared-folder row applies seq floor, not ungated network | +| `spawn_file_meta_reencrypt` twin caller | FUSE implementation phase | Both `rename.rs` and `platform/windows/write_ops.rs` updated; Windows CI passes | +| Cold-node filename leak on rotation tail | Rotation engine phase | Eager-walk invariant documented; no lazy-walk variant introduced | --- ## Sources -- [IPFS IPNS Concepts](https://docs.ipfs.tech/concepts/ipns/) -- DHT expiry (48h), TTL, republishing behavior -- [Measuring IPNS Performance on the Public Amino DHT](https://www.probelab.network/blog/ipns-performance-amino-dht) -- Median 11s retrieval latency, 100% retrieval success rate -- [Someguy - Delegated Routing V1 server](https://github.com/ipfs/someguy) -- Self-hosted delegated routing, proxies to DHT and IPNI -- [Kubo Delegated Routing docs](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md) -- Routing.Type auto/dht/custom configuration -- [IPFS Public Utilities](https://docs.ipfs.tech/concepts/public-utilities/) -- delegated-ipfs.dev as public good endpoint -- [Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- IPIP-476, IPIP-513, Someguy improvements -- [IPFS Pinning Service API spec](https://ipfs.github.io/pinning-services-api-spec/) -- Async pinning, standard API for BYO integration -- [Multiple users publish to IPNS at the same time](https://github.com/ipfs/kubo/issues/8433) -- Concurrent publish conflict, sequence number semantics -- [IPNS Record and Protocol spec](https://specs.ipfs.tech/ipns/ipns-record/) -- Sequence number comparison, record validation -- [Empirical study on performance overhead of code instrumentation (2025)](https://www.sciencedirect.com/science/article/pii/S0164121225002420) -- Up to 8.4% throughput reduction, 20-49% latency increase from instrumentation -- [How to Reduce OpenTelemetry Performance Overhead in Production](https://oneuptime.com/blog/post/2026-02-06-reduce-opentelemetry-performance-overhead-production/view) -- Sampling strategies, batch processing -- [OpenTelemetry NestJS Implementation Guide](https://signoz.io/blog/opentelemetry-nestjs/) -- NestJS-specific OTel setup and sampling -- CipherBox codebase: `apps/api/src/ipns/ipns.service.ts`, `apps/api/src/ipns/delegated-routing.client.ts`, `apps/api/src/republish/republish.service.ts`, `apps/web/src/services/ipns.service.ts`, `apps/web/public/recovery.html` -- CipherBox project context: `.planning/PROJECT.md`, `.planning/codebase/CONCERNS.md`, `.planning/todos/pending/` (IPNS alternatives, BYO-IPFS, move rootFolderKey) +- `.planning/design/2026-06-26-sharing-read-keychaining-design.md` — primary design source; findings CRIT-1, M1, HIGH-3, HIGH-4, MED-5, MED-6, m1–m4 and Section 7.3 test strategy +- `docs/adr/0001-write-revocation-full-ed25519-rotation.md` — write-revocation rationale and consequences +- `docs/adr/0002-read-revocation-protects-future-content-only.md` — content-key rotation scope and accepted residuals +- `CONTEXT.md` — glossary and counter disambiguation (`generation` vs `keyEpoch` vs `sequenceNumber`) +- `docs/METADATA_EVOLUTION_PROTOCOL.md` — schema versioning discipline; cross-platform round-trip requirements +- `docs/DATABASE_EVOLUTION_PROTOCOL.md` — migration discipline; conditional-UPDATE patterns +- `CLAUDE.md` (project) — critical security rules; `Critical Security Rules` section +- Project memory (`MEMORY.md`) — Phase-60 null-column regressions; winfsp nesting trap; sdk-core barrel coverage; SDK E2E as only real publish/resolve gate; zeroization bug; desktop E2E dispatch-gating; web vitest `.test.ts` only; folderTree/Zustand desync class --- -_Pitfalls research for: CipherBox v1.1 IPFS Infrastructure_ -_Researched: 2026-03-07_ +_Pitfalls research for: v2.0 Metadata and Sharing Refactor (node/v3 read key-chaining)_ +_Researched: 2026-06-27_ diff --git a/.planning/research/SUMMARY.md b/.planning/research/SUMMARY.md index d720bfb65c..d8fd206147 100644 --- a/.planning/research/SUMMARY.md +++ b/.planning/research/SUMMARY.md @@ -1,187 +1,65 @@ -# Project Research Summary +# Research Summary: v2.0 Metadata and Sharing Refactor -**Project:** CipherBox v1.1 -- IPFS Infrastructure -**Domain:** IPFS/IPNS reliability, database minimization, BYO-IPFS node support, performance baselines -**Researched:** 2026-03-07 -**Confidence:** HIGH +**Synthesized:** 2026-06-27 +**Dimensions researched:** Architecture + Pitfalls only (Stack + Features intentionally skipped — design is implementation-ready) +**Confidence:** HIGH — all cited symbols verified against live codebase (zero material drift) ## Executive Summary -CipherBox v1.1 is an infrastructure-hardening milestone, not a feature milestone. The four workstreams -- IPNS reliability, database minimization, BYO-IPFS, and performance baselines -- aim to transform CipherBox from "IPFS as a storage backend with database fallbacks" to "IPFS-native with the database serving only coordination and auth." The dominant finding across all four research files is that the existing stack already has the capabilities needed. The Kubo v0.34.0 node can resolve IPNS records locally via its DHT; the `IpfsProvider` interface already abstracts pin/unpin/get for provider extension; `prom-client` is installed with an established `MetricsService` pattern; and TypeORM handles all necessary migrations. **Zero new npm dependencies are required for this entire milestone.** +CipherBox v2.0 replaces a database-driven per-item key fan-out model (`share_keys` table, `O(items × recipients)` rows) with metadata-driven read key-chaining, and closes two confirmed revocation gaps: lazy/unsound read-revocation and un-rotatable write delegation. Scope is locked to **Tier 1** (unified `Node` schema, read key-chain navigation, resumable read-rotation engine) and **Tier 2** (write-revocation via full Ed25519 rotation — ADR 0001, and the resolve/republish/TEE contract rewrite). **Tier 3** (capability-layer TTL/op-caps) is explicitly out of scope. -The recommended approach is a five-phase execution with strict ordering (Phases 18-22). Phase 18 (performance instrumentation) is zero-risk and establishes server-side baselines before any architectural changes. Phase 19 (IPNS reliability) inverts the resolution model to DB-first with async Kubo DHT verification via self-hosted Someguy, eliminating the `delegated-ipfs.dev` dependency. Phase 20 (database minimization) moves `encryptedRootFolderKey` into an IPFS blob via a versioned vault blob v2 format, with dual-write migration and permanent DB fallback. Phase 21 (BYO-IPFS) adds a `RemotePinningProvider` speaking the standard IPFS Pinning Service API, with dual-pin strategy (always pin to CipherBox node, best-effort mirror to user's node). Phase 22 (performance baselines completion) adds client-side timing instrumentation, end-to-end journey timing, k6 load tests, and capacity documentation after all features are stable. +The design is implementation-ready — three adversarial reviews, two grilling sessions, and two ratified ADRs; no design decisions remain open except the three §9.2 questions flagged below. Stack and Features research were deliberately skipped because the design already pins them. -The primary risk is in Phase 3: moving rootFolderKey to IPFS creates a hard dependency on IPNS resolution for login. All four research files flag this independently. The mitigation is clear -- keep the DB copy as a permanent fallback, never make IPNS the sole access path for the root folder key. Secondary risks include sequence number divergence during routing provider migration (Phase 2) and quota tracking becoming unenforceable for BYO-IPFS users (Phase 4). Both have well-defined prevention strategies documented in PITFALLS.md. +The build order is strictly dependency-constrained across 8 layers: crypto primitive → core keystone → sdk-core rotation engine → sdk write/bin/invite → api schema+publish-gate → tee-worker contract → web → fuse/winfsp. The **AAD-bound seal primitive must land first** because a byte-encoding mismatch between TS and Rust is a **silent total decryption failure** — no error, just broken unseals across every node. **`packages/core` is the keystone**: nothing below it typechecks until `Node`/`SealedChildRef`/`PublishedNode` exist and `dist/` is rebuilt. -## Key Findings +## The Three Silent-Failure Risks (must be test-gated before their phase closes) -### Recommended Stack +These fail with NO runtime error — the worst class for this refactor: -The milestone requires zero new npm dependencies. Every capability is built using existing libraries, configuration changes, and new provider implementations. The only external tooling addition is Grafana k6 v1.0 (a standalone Go binary, not an npm package) for load testing scripts. +1. **CRIT-1 — content-key rotation.** Rotating a file node without minting a new `fileKey'` leaves the revoked reader able to decrypt the next content version. Gate: §7.3 test 2. +2. **M1 — generation downgrade defense persistence.** The `{nodeId → highestGeneration}` high-water must be **durable** (IndexedDB/sqlite); stored in memory it evaporates on restart and a colluding relay replays the pre-rotation record. Gate: §7.3 test 5 (must survive restart). +3. **Republisher sequence increment.** The TEE republisher must STOP incrementing `sequenceNumber` (`apps/tee-worker/src/routes/republish.ts:79` `+ 1n` confirmed) — a re-signed stale pre-rotation CID at a forward sequence dominates the rotation publish. Gate: §7.3 test 12. -**Core technologies (all existing):** +Plus the top silent-**data-loss** risk: -- **Kubo v0.34.0 (recommend upgrade to v0.40.1):** Self-hosted IPFS node already participating in the Amino DHT. Supports `Gateway.ExposeRoutingAPI` and `Ipns.UsePubsub` for local IPNS resolution. v0.40.1 makes routing API default and improves IPNS-over-PubSub. -- **`prom-client` ^15.1.3:** Already installed with `MetricsService` pattern established. Extend with 4 new histograms for IPFS/IPNS latency tracking. -- **TypeORM ^0.3.28 + PostgreSQL 16.x:** Handles all migration needs. 2-3 new migrations (ADD COLUMN, ALTER COLUMN, CREATE TABLE for user IPFS config). -- **IPFS Pinning Service API (HTTP standard):** Vendor-agnostic OpenAPI spec with 4 endpoints. Implemented by Pinata, Filebase, web3.storage, IPFS Cluster. No SDK needed -- native `fetch` is sufficient. -- **k6 v1.0+ (dev tooling only):** Standalone load test runner with native TypeScript support. Install via `brew install k6`. Not committed to repo. +4. **HIGH-4 — add-during-rotation.** The 409-retry path must re-fetch and re-merge `SealedChildRef`s, not re-seal from stale in-memory `children[]`; otherwise a concurrent upload is silently dropped. Gate: §7.3 test 4. -### Expected Features +## Suggested Phase Decomposition (8 phases, dependency-ordered) -**Must have (table stakes):** +1. **Crypto Primitive** — `sealAesGcmAad`/`unsealAesGcmAad` + `buildNodeAad` (TS) + byte-identical Rust twin + committed cross-language KAT (all 4 role bytes; frozen byte encoding). Self-contained, no consumers break. Gate: tests 6 (AAD transplant) + 7 (TS↔Rust KAT). +2. **Core Keystone** — unified `Node`/`SealedChildRef`/`PublishedNode` + codecs (two sealed bodies, content self-seal, structured write chain, `versionFloor`). Gate: all consumers typecheck after `dist/` rebuild. Critical-path bottleneck — nothing below typechecks until this lands. +3. **sdk-core Rotation Engine** — read-chain navigation + `rotateReadFromNode`/`rotateOne`/`verifySubtreeClean` in **named files (not a fat `index.ts` barrel** — coverage excludes barrels). Must include CRIT-1, M1, HIGH-3, HIGH-4 as success criteria. Gate: tests 1–5, 9; SDK E2E pass. +4. **sdk Write/Bin/Invite** — `shared-write.ts` rewrite (structured write-body, (c) full rotation, role `0x04`); delete `addShareKeys`/`reWrapForRecipients`; `bin/*` re-link (delete `originalFolderKeyEncrypted` re-encrypt path); invite claim re-wrap (delete `encryptedChildKeys[]` JSONB fan-out). Gate: tests 10 (bin restore), 11 (invite claim). +5. **API Schema + Publish Gate** — delete `share_keys`; slim `shares` (`readDescriptorRef`/`writeDescriptorRef`); rename `folder_ipns` → `ipns_records`; **drop `folder_ipns.public_key`** (null-row footgun behind two Phase-60 regressions); collapse `ipns_republish_schedule` duplicated columns; **atomic conditional-UPDATE publish CAS**; tombstone state + publish-gate rejection + resolve fail-closed fall-through (case-split: expected-null shared-folder rows apply seq floor, CID-mismatch fails closed); server-side generation gate. Run `pnpm api:generate`, commit regenerated client (`check-api-client.sh` pre-commit gate). Gate: tests 13, 15, 16, 20. +6. **TEE Worker Contract** — lease-renewer rewrite: receive marshaled record, verify signature, extend EOL only, **no CID origination, no sequence increment**; internal epoch derivation; name↔key binding; migration durability. Round-trip the TEE/republish E2E. Gate: tests 12, 17, 18, 19. +7. **Web** — replace `executeLazyRotation` with `rotateReadFromNode`; drop per-mutation fan-out; reconcile `folderTree` against `sequenceNumber` before publishes; durable IndexedDB generation + seq high-water. Note: web vitest only runs `*.test.ts` (not `.spec.ts`). Gate: test 5 (durable survives restart), test 13. +8. **FUSE + WinFsp** — symmetric child-key unwrap; delete `spawn_file_meta_reencrypt` from **both callers** (`write_ops/.../rename.rs` AND `platform/windows/write_ops.rs`); add `rotateReadFromNode`; unify scope-exit; **grant-root awareness** in `delete`/`rename`/`move` (net-new); durable client floors; strict-verify each republish (recover Ed25519 pubkey from k51 name via `publicKeyFromIpnsName`, never the dropped column); `Node` as a real Rust enum. Budget a Windows CI round-trip (winfsp can't compile on macOS; watch the `super::` vs `super::super::` nesting trap). Gate: test 21 (`Cargo Check & Test (Windows)` authoritative); desktop E2E is dispatch-gated — trigger explicitly. -- Reliable IPNS resolution (<2s, >99.5% availability) -- current `delegated-ipfs.dev` has documented 502s and stale records -- DB-cached CID fallback with sequence number comparison -- already implemented, needs to become the primary resolution path -- IPNS publish/resolve latency monitoring -- counters exist, need duration histograms -- API endpoint response time baselines with p50/p95/p99 targets -- Graceful degradation when IPFS/IPNS is slow -- timeout + fallback pattern, partially implemented +## Test Strategy Thread (§7.3 — must-pass-before-merge) -**Should have (differentiators):** +The design's 21-item test list maps onto the phases above. The crash-safety/resume suite (test 1) and the three silent-revocation-bypass tests (2, 5, 12) plus the silent-data-loss test (4) are the non-negotiable merge gates. `tests/sdk-e2e` is the only real client→API IPNS publish/resolve round-trip — extend it with abort-and-resume cases. Keep checker subagents to static analysis only (no concurrent vitest — RAM starvation). -- Move rootFolderKey to IPFS vault record -- server retains DB copy as permanent fallback but primary access shifts to IPFS blob, reducing server's role toward zero-knowledge relay -- BYO-IPFS node support via Pinning Service API -- unique among encrypted storage apps, enables self-sovereignty over data persistence -- End-to-end user journey performance baselines -- login-to-vault, upload-to-visible timing -- IPFS/IPNS latency histograms with per-operation breakdown in Prometheus +## Drift Report (design citations vs live code) -**Defer (v1.2+):** +- `decryptFileMetadata` is ~line 231 in `packages/core/src/file/metadata.ts` (design cites 232) — immaterial. +- All other 17 cited symbols verified present: TEE `+ 1n` (`republish.ts:79`), `unenrollIpns` schedule-only delete (`republish.service.ts:257`), `folder_ipns.public_key` nullable (`Buffer | null`, entity ~line 63), `encryptedChildKeys` JSONB on `share_invites`, `originalFolderKeyEncrypted` re-encrypt path (`packages/sdk/src/bin/index.ts:688`), `parseCachedRecord`-null fall-through (`ipns.service.ts:504`, case-dependent two-branch fix). -- CRDT-based share discovery via IPNS inbox -- research-only in v1.1, architecturally desirable but premature -- Migrate device registry off DB -- requires solving approval handshake without server coordination -- Eliminate `pinned_cids` table -- requires alternative quota tracking -- Client-direct IPFS upload mode -- CORS issues, breaks quota tracking and conflict detection +## Sub-phase Research Flags -### Architecture Approach +- **Phase 5 (API):** TypeORM FK constraint map for the `folder_ipns` → `ipns_records` rename — the FK references must be inspected (against staging DB schema) before writing the migration; all referencing tables migrate atomically. +- **Phase 8 (FUSE):** grant-root scope computation algorithm is net-new and under-specified in the design — needs a plan-time design pass. +- Skip research: Phase 1 (additive AES primitive), Phase 2 (schema fully specified), Phase 6 (surgical TEE rewrite, fully specified in design §6.2–6.7). -The architecture follows a "DB-first, IPFS-verify" pattern for IPNS resolution, a versioned blob format for metadata evolution, a provider factory pattern for BYO-IPFS extensibility, and additive histogram instrumentation for performance baselines. The key insight is that the DB already serves as the reliable source for IPNS CIDs -- making this explicit (instead of treating it as a fallback) simplifies the architecture and eliminates the external `delegated-ipfs.dev` dependency without adding new infrastructure. +## Open Questions for Owning Phases (design §9.2) -**Major components:** +- **Q1 — Co-writer offline handling** under (c): a co-writer offline during write-key rotation can't write until re-fetch. Accept as explicit, or add grace/notification? → owning phase: Web (Phase 7). +- **Q2 — Rotation host for pure-web users:** eager million-node rotation is owner-online and resumable; desktop (FUSE) is the natural host. Is a long, chunked, multi-session web rotation acceptable for a large revoke? → document in sdk-core (Phase 3). +- **Q3 — Write-recipient-vs-owner sub-share authority:** when a write-recipient deletes/moves a node the owner independently sub-shared, the unlink and the revocation split across two principals. Decide the authority model + acceptable exposure window. → owning phases: Web (Phase 7) + FUSE delete path (Phase 8). -1. **KuboIpnsClient** -- New NestJS injectable wrapping Kubo's RPC API (`/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import`). Handles native DHT resolution and Kubo-native publishing for the TEE republish path. -2. **Vault Blob v2 Format** -- Extends the root IPNS blob with a version byte and ECIES-wrapped `encryptedRootFolderKey` header, enabling client-side key extraction without a DB round-trip. Version-aware reading allows graceful fallback to blob v1. -3. **Provider Factory + DualPinProvider** -- Per-user `IpfsProviderFactory` that returns either the default `LocalProvider` or a `DualPinProvider` wrapping both `LocalProvider` (always) and `UserCustomProvider` (best-effort mirror to user's node). -4. **MetricsService Extensions** -- 4 new Prometheus histograms: IPNS resolve duration, IPNS publish duration, IPFS operation duration, TEE republish batch duration. All use the existing `prom-client` infrastructure. +## Watch Out For (top execution traps from project history) -### Critical Pitfalls - -1. **Sequence number divergence during routing provider migration** -- The new Kubo DHT endpoint may have no records initially. Run the new provider in read-parallel mode for 48+ hours before cutting writes. Dual-write to old and new providers during transition. Monitor for 409 Conflict error spikes. - -2. **rootFolderKey on IPNS creates login-critical IPNS dependency** -- Moving rootFolderKey exclusively to IPFS means login fails when IPNS is down. The mitigation is absolute: keep the DB copy as the primary source for login, use the IPFS copy for recovery tool independence. This is a "both, not either" situation. Never drop the DB column. - -3. **DHT record expiry during routing provider migration** -- IPNS records expire from the DHT after 48 hours regardless of the validity field. If both providers are misconfigured during a weekend deployment, ALL records expire simultaneously. Build an emergency republish endpoint that bypasses the 6-hour schedule. - -4. **BYO-IPFS bypasses server-side optimistic concurrency** -- If users publish IPNS records directly to their node (bypassing the API), the server cannot check sequence numbers. The fix: BYO-IPFS affects ONLY where data is pinned, never how metadata is published. All IPNS publishes must still go through the CipherBox API. - -5. **Quota tracking becomes unenforceable with BYO-IPFS** -- Server cannot meter uploads it does not relay. For BYO users, skip server-side quota enforcement and require client-reported CID sizes. Mark `pinned_cids` entries with a `provider` column to distinguish server-pinned from BYO-pinned. - -## Implications for Roadmap - -Based on research, suggested phase structure: - -### Phase 1: Performance Instrumentation - -**Rationale:** Zero risk to existing functionality. Purely additive. Establishes "before" measurements that Phase 2-4 can be compared against. Without baselines, we cannot prove that subsequent phases improve performance or detect regressions. -**Delivers:** 4 new Prometheus histograms (IPNS resolve/publish duration, IPFS operation duration, TEE republish batch duration), client-side timing utility, baseline measurements document, k6 load test scripts. -**Addresses:** Table stakes features -- IPNS publish latency monitoring, API endpoint baselines. -**Avoids:** Pitfall P7 (instrumentation overhead) -- verify overhead < 5% on P95 latency with and without instrumentation. -**Estimated scope:** Small. ~10 modified files, no new entities or migrations. - -### Phase 2: IPNS Resolution Improvement - -**Rationale:** Fixes the most critical reliability issue and is the gating dependency for Phase 3. The current `delegated-ipfs.dev` dependency causes 502 errors and stale records. Must be resolved before making IPNS a login-critical path. -**Delivers:** `KuboIpnsClient` for native Kubo DHT resolution, DB-first resolution strategy, `delegated-ipfs.dev` demoted to disabled-by-default fallback, Kubo config with `Ipns.UsePubsub: true`, updated recovery tool resolution chain. -**Addresses:** Table stakes -- reliable IPNS resolution, graceful degradation. -**Avoids:** Pitfall P1 (sequence number divergence) -- run dual-provider for 48+ hours, compare sequence numbers across providers. Pitfall P5 (DHT record expiry) -- never take old provider offline until new one has processed a full republish cycle. -**Estimated scope:** Medium. 1 new file, ~5 modified files, Kubo config change, recovery tool update. - -### Phase 3: Database Minimization (rootFolderKey to IPFS) - -**Rationale:** Requires reliable IPNS resolution from Phase 2. This is the highest-value zero-knowledge improvement: the server stores zero crypto material after migration. The vault blob v2 format is a breaking metadata change requiring careful migration across web, desktop, and recovery tool. -**Delivers:** Vault blob v2 format with embedded `encryptedRootFolderKey`, version-aware blob reading, dual-write migration strategy, `encryptedRootIpnsPrivateKey` deprecation, updated recovery tool, metadata schema version bump per METADATA_EVOLUTION_PROTOCOL. -**Addresses:** Differentiator -- server stores zero crypto material, true zero-knowledge relay. -**Avoids:** Pitfall P2 (rootFolderKey on IPNS) -- keep DB copy as permanent fallback for login, use IPFS copy for recovery tool independence. Pitfall P3 (share discovery on IPFS) -- explicitly do NOT migrate shares, keep sharing graph in DB. -**Estimated scope:** Large. New type definitions, serialization code, changes across 3 clients (web, desktop, recovery). - -### Phase 4: BYO-IPFS Node Support - -**Rationale:** Largest surface area but most independent feature. Benefits from stable IPNS resolution (Phase 2) and reduced DB dependency (Phase 3) but does not strictly require them. The `RemotePinningProvider` is additive -- it mirrors pins without replacing the local provider. -**Delivers:** `UserCustomProvider` (Kubo RPC + Pinning Service API), `DualPinProvider` wrapper, `ProviderFactory` for per-user resolution, `user_ipfs_config` entity and migration, IPFS config CRUD API endpoints, connection test endpoint, Settings page UI for IPFS node configuration, BYO-specific Prometheus metrics. -**Addresses:** Differentiator -- BYO-IPFS node support, unique among encrypted storage apps. -**Avoids:** Pitfall P4 (BYO bypasses concurrency) -- all IPNS publishes still go through API, BYO affects only pinning. Pitfall P6 (quota tracking) -- skip server-side quota for BYO users, require client-reported CID sizes, add `provider` column to `pinned_cids`. -**Estimated scope:** Large. New entity, migration, 4+ new provider files, UI component, multiple API endpoints. - -### Phase Ordering Rationale - -- **Phase 1 before everything:** Baselines must exist before changes, or you cannot measure impact. Zero dependencies, zero risk. -- **Phase 2 before Phase 3 (hard dependency):** Moving rootFolderKey to IPFS makes IPNS resolution login-critical. IPNS must be reliable first. All four research files independently flag this as the most important ordering constraint. -- **Phase 3 before Phase 4 (soft dependency):** Not strictly required, but Phase 3 reduces DB dependency which aligns with BYO-IPFS goals. Phase 4's design work can proceed in parallel with Phase 3's implementation. -- **Phase 4 last:** Largest surface area, most new code, benefits from all prior infrastructure improvements. - -### Research Flags - -Phases likely needing deeper research during planning: - -- **Phase 3 (Database Minimization):** Migration protocol edge cases -- blob v2 format across 3 clients, dual-write/dual-read window management, recovery tool independence, desktop (Rust) blob parsing. This is the riskiest phase. -- **Phase 4 (BYO-IPFS):** Per-user provider routing in NestJS DI, IPFS Pinning Service API real-world testing, auth token storage model (ECIES-wrapped vs server-encrypted), Settings UI design. - -Phases with standard patterns (skip research-phase): - -- **Phase 1 (Performance Instrumentation):** Established `prom-client` + `MetricsService` pattern. k6 is well-documented. Purely additive. -- **Phase 2 (IPNS Resolution):** Kubo RPC API is documented, DB-first resolution is architecturally straightforward. The main risk (sequence number divergence during transition) is an operational concern, not a research gap. - -## Confidence Assessment - -| Area | Confidence | Notes | -| ------------ | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Stack | HIGH | Zero new npm dependencies. All capabilities use existing libraries. Kubo API contracts verified against official docs. | -| Features | HIGH for IPNS/perf, MEDIUM for DB migration and BYO-IPFS | IPNS reliability and performance monitoring are well-documented domains. DB-to-IPFS migration has app-specific edge cases. BYO-IPFS Pinning Service API is well-specified but per-user provider routing patterns are sparse. | -| Architecture | HIGH | Existing codebase analyzed in detail. DB-first resolution, provider factory, blob versioning are all established patterns. Build order validated against dependency graph. | -| Pitfalls | HIGH | 6 critical pitfalls identified with specific prevention strategies, warning signs, and recovery plans. Phase-to-pitfall mapping is complete. | - -**Overall confidence:** HIGH - -### Gaps to Address - -- **rootFolderKey migration dual-write window:** How long should the dual-write period last? What happens to users who never log in during the window? Need a forced migration strategy (background job that reads DB key, writes blob v2, publishes to IPNS) for dormant accounts. -- **BYO-IPFS auth token storage model:** STACK.md recommends server-side encryption (server needs to decrypt to call user's node). ARCHITECTURE.md implements this. PITFALLS.md warns about server compromise exposing tokens. The tradeoff (server sees token but not plaintext content) needs explicit acceptance and documentation. -- **Kubo version decision:** v0.34.0 works but v0.40.1 is recommended. The upgrade should happen before Phase 2 starts but is not blocking. Need to validate that Kubo v0.40.1 does not introduce breaking changes for the existing `LocalProvider` calls. -- **CRDT inbox for shares:** Research-only this milestone. Need to document the CRDT approach as a design RFC during Phase 3 work so it feeds into v1.2 planning. -- **Recovery tool independence:** The recovery tool currently resolves IPNS via `delegated-ipfs.dev` directly from the browser. Phases 2 and 3 both modify its behavior. Need to verify it works WITHOUT the CipherBox API running after all changes. -- **IPNS routing approach reconciliation:** STACK.md recommends Kubo's `Gateway.ExposeRoutingAPI`, FEATURES.md recommends self-hosted Someguy, ARCHITECTURE.md recommends DB-first with Kubo RPC `/api/v0/name/resolve`. Adopt the ARCHITECTURE.md approach (DB-first with async Kubo DHT verification) as the definitive strategy -- it is the most robust because it makes the already-reliable DB the primary path and uses Kubo DHT only for background verification. - -## Sources - -### Primary (HIGH confidence) - -- [Delegated Routing V1 HTTP API Spec](https://specs.ipfs.tech/routing/http-routing-v1/) -- IPNS endpoint contract -- [IPFS Pinning Service API Spec](https://ipfs.github.io/pinning-services-api-spec/) -- BYO-IPFS integration standard -- [Kubo RPC API v0 Reference](https://docs.ipfs.tech/reference/kubo/rpc/) -- `/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import` -- [Kubo Configuration Reference](https://github.com/ipfs/kubo/blob/master/docs/config.md) -- `Ipns.UsePubsub`, `Routing.Type`, `Gateway.ExposeRoutingAPI` -- [Kubo v0.34.0](https://github.com/ipfs/kubo/releases/tag/v0.34.0) and [v0.40.0](https://github.com/ipfs/kubo/releases/tag/v0.40.0) Release Notes -- [Someguy GitHub](https://github.com/ipfs/someguy) -- Self-hosted delegated routing reference -- [Grafana k6 1.0 Release](https://grafana.com/blog/grafana-k6-1-0-release/) -- Load testing -- [prom-client GitHub](https://github.com/siimon/prom-client) -- Prometheus client for Node.js -- [IPNS Record and Protocol spec](https://specs.ipfs.tech/ipns/ipns-record/) -- Sequence number semantics - -### Secondary (MEDIUM confidence) - -- [ProbeLab IPFS KPIs](https://www.probelab.io/ipfs/kpi/) and [Week 07, 2026 Results](https://discuss.ipfs.tech/t/probelabs-notable-ipfs-performance-results-week-07-2026/20048) -- DHT performance baselines -- [IPIP-0379: Delegated IPNS HTTP API](https://specs.ipfs.tech/ipips/ipip-0379/) -- IPNS delegation spec -- [IP Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- IPFS ecosystem context -- [Measuring IPNS Performance on the Public Amino DHT](https://www.probelab.network/blog/ipns-performance-amino-dht) -- Median 11s retrieval latency reference -- [Empirical study on performance overhead of code instrumentation (2025)](https://www.sciencedirect.com/science/article/pii/S0164121225002420) -- Instrumentation overhead benchmarks - -### Codebase (HIGH confidence) - -- `apps/api/src/ipns/ipns.service.ts` -- Current IPNS resolution logic with DB fallback -- `apps/api/src/ipns/delegated-routing.client.ts` -- Current routing client (3 retries, 10s timeout) -- `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` -- 3-method provider abstraction -- `apps/api/src/metrics/metrics.service.ts` -- Existing Prometheus infrastructure -- `apps/api/src/republish/republish.service.ts` -- TEE republish via delegated routing -- `apps/web/public/recovery.html` -- Standalone recovery tool with direct IPNS resolution -- `docs/METADATA_SCHEMAS.md` and `docs/METADATA_EVOLUTION_PROTOCOL.md` -- Schema migration rules - ---- - -_Research completed: 2026-03-07_ -_Ready for roadmap: yes_ +- AAD byte-encoding drift TS↔Rust = silent total decryption failure → KAT is Phase 1's merge gate. +- `folder_ipns.public_key` null-row footgun → drop the column; recover pubkey from k51 name. +- folderTree/Zustand vs SDK desync ("Folder not loaded" class) → reconcile before rotation publishes. +- Zeroization of caller-owned/reused buffers breaks SDK E2E → zero only at terminal owner. +- winfsp can't compile on macOS; desktop E2E is dispatch-gated → explicit CI round-trips as phase completion criteria, not post-merge. diff --git a/.planning/research/_v1.1-archive/ARCHITECTURE.md b/.planning/research/_v1.1-archive/ARCHITECTURE.md new file mode 100644 index 0000000000..0f5c18f039 --- /dev/null +++ b/.planning/research/_v1.1-archive/ARCHITECTURE.md @@ -0,0 +1,979 @@ +# Architecture: IPFS Infrastructure v1.1 + +**Domain:** IPNS reliability, database minimization, BYO-IPFS, performance baselines +**Researched:** 2026-03-07 +**Confidence:** HIGH (existing codebase analyzed; Kubo APIs verified against official docs) + +--- + +## Table of Contents + +1. [Existing Architecture Summary](#1-existing-architecture-summary) +2. [IPNS Resolution: Replace delegated-ipfs.dev](#2-ipns-resolution-replace-delegated-ipfsdev) +3. [Database Minimization: Migration Path](#3-database-minimization-migration-path) +4. [BYO-IPFS Node Support](#4-byo-ipfs-node-support) +5. [Performance Instrumentation](#5-performance-instrumentation) +6. [Component Boundary Map](#6-component-boundary-map) +7. [Data Flow Changes](#7-data-flow-changes) +8. [Suggested Build Order](#8-suggested-build-order) +9. [Sources](#9-sources) + +--- + +## 1. Existing Architecture Summary + +### Current IPNS Flow (What Changes) + +```text +Client API delegated-ipfs.dev Kubo + | | | | + |-- sign IPNS record ---->| | | + | |-- upsert folder_ipns (DB) --->| | + | |-- PUT /routing/v1/ipns/:name ->| | + | |<-- 200 OK or 502 -------------| | + |<-- { sequenceNumber } --| | | + | | | | + |-- resolve IPNS -------->| | | + | |-- GET /routing/v1/ipns/:name ->| | + | |<-- record bytes or 502 --------| | + | |-- fallback: query folder_ipns --| | + |<-- { cid, seqNum } -----| | | +``` + +**Key problem:** `delegated-ipfs.dev` is the sole IPNS resolution path from the API. The DB-cached CID is the fallback, but this means the DB is doing double duty as both CID cache and the reliable resolution source. The external service adds latency (10s timeout, 3 retries with backoff) and has documented 502 reliability issues. + +### Current Database Tables (What Shrinks) + +| Table | Rows/User | Purpose | Can Migrate to IPFS? | +| ------------------------- | ------------------ | --------------------------------------------------------------------------------- | ------------------------------------------------------ | +| `users` | 1 | UUID, publicKey | NO -- identity anchor | +| `auth_methods` | 1-5 | Auth provider links | NO -- server-side auth | +| `refresh_tokens` | 1-3 | JWT sessions | NO -- server-side auth | +| `vaults` | 1 | encryptedRootFolderKey, encryptedRootIpnsPrivateKey, rootIpnsName, ownerPublicKey | PARTIALLY -- rootFolderKey to IPFS | +| `folder_ipns` | N folders+files | latestCid, sequenceNumber, encryptedIpnsPrivateKey, keyEpoch | YES -- but serves as CID cache and concurrency tracker | +| `pinned_cids` | N files | CID, sizeBytes (quota tracking) | NO -- server enforces quota | +| `ipns_republish_schedule` | N folders+files | TEE republish state | NO -- TEE orchestration is server-side | +| `shares` | N shares | Sharer, recipient, encryptedKey, revocation | FUTURE -- CRDT inbox research | +| `share_keys` | N file/folder keys | Per-item encrypted keys for shares | FUTURE -- CRDT inbox research | +| `share_invites` | sparse | Link sharing tokens | NO -- server-validated tokens | +| `device_approvals` | sparse | MFA device approval state | NO -- short-lived server state | +| `tee_key_state` | 1 | TEE epoch tracking | NO -- server-side TEE coordination | +| `tee_key_rotation_log` | sparse | Rotation audit trail | NO -- admin audit | + +### Current IPFS Provider Abstraction + +The existing `IpfsProvider` interface is minimal (3 methods): + +```typescript +interface IpfsProvider { + pinFile(data: Buffer, metadata?: Record): Promise<{ cid: string; size: number }>; + unpinFile(cid: string): Promise; + getFile(cid: string): Promise; +} +``` + +Only `LocalProvider` (Kubo) exists. The interface is injected via `IPFS_PROVIDER` DI token in `IpfsModule.forRootAsync()`, configured from env vars `IPFS_LOCAL_API_URL` and `IPFS_LOCAL_GATEWAY_URL`. + +--- + +## 2. IPNS Resolution: Replace delegated-ipfs.dev + +### 2.1 Recommendation: DB-First Resolution + Kubo DHT Verification + +**Strategy:** Invert the current resolution priority. Make the DB-cached CID the primary resolution source (it is already written synchronously on every publish). Use the self-hosted Kubo node's `/api/v0/name/resolve` endpoint for background DHT verification. Demote `delegated-ipfs.dev` to a disabled-by-default fallback. + +**Why this approach:** + +1. **Eliminates external dependency.** The self-hosted Kubo node already participates in the IPFS DHT. It can resolve IPNS names directly without delegated-ipfs.dev. +2. **DB cache is already the reliable source.** The `IpnsService.resolveRecord()` method (lines 290-350 of `ipns.service.ts`) already prefers DB cache when its sequence number is higher than the network result. Making DB-first explicit simplifies the architecture. +3. **Kubo DHT resolution has improved.** Kubo 0.38+ has sweep providers with 97% fewer DHT lookups for large-scale operations. IPNS TTL defaults dropped from 1 hour to 5 minutes in recent releases, so changes propagate faster. +4. **PubSub for same-node resolution.** Kubo supports `Ipns.UsePubsub` for near-instant IPNS record propagation between connected peers. When both the publishing client and the TEE republisher connect through the same Kubo node, PubSub makes resolution instantaneous. + +### 2.2 Architecture Change + +**Before (current):** + +```text +IpnsService.resolveRecord() + -> DelegatedRoutingClient.resolve() -> https://delegated-ipfs.dev (primary, 10s timeout) + -> folder_ipns DB query (fallback on 502/timeout) + -> Compare sequence numbers, return highest +``` + +**After:** + +```text +IpnsService.resolveRecord() + -> folder_ipns DB query (primary, <5ms) + -> Return DB result immediately to caller + -> Async: KuboIpnsClient.resolve() -> http://kubo:5001/api/v0/name/resolve (background) + -> If DHT has higher sequence number, update DB cache + -> If DHT fails, no-op (DB is authoritative for our records) +``` + +### 2.3 New Component: KuboIpnsClient + +Wraps Kubo's RPC API for IPNS operations: + +```typescript +// apps/api/src/ipns/kubo-ipns.client.ts +@Injectable() +export class KuboIpnsClient { + constructor(private readonly configService: ConfigService) { + this.kuboApiUrl = configService.get('IPFS_LOCAL_API_URL', 'http://localhost:5001'); + } + + /** + * Resolve IPNS name via Kubo DHT. + * POST /api/v0/name/resolve?arg=&dht-timeout=5s&nocache=true + * Returns the resolved /ipfs/ path or null if not found. + */ + async resolve(ipnsName: string): Promise<{ path: string } | null> { + /* ... */ + } + + /** + * Publish via Kubo's native IPNS publisher (for TEE republish path). + * Requires key imported into Kubo's keystore first. + * POST /api/v0/name/publish?arg=/ipfs/&key=&ttl=5m + */ + async publish(keyName: string, cid: string, options?: { ttl?: string }): Promise { + /* ... */ + } + + /** + * Import an Ed25519 private key into Kubo's keystore. + * POST /api/v0/key/import?arg=&format=libp2p-protobuf-cleartext + * Body: multipart/form-data with key bytes + */ + async importKey(keyName: string, privateKeyBytes: Uint8Array): Promise { + /* ... */ + } + + /** + * List keys in Kubo's keystore (for cleanup). + * POST /api/v0/key/list + */ + async listKeys(): Promise { + /* ... */ + } + + /** + * Remove a key from Kubo's keystore. + * POST /api/v0/key/rm?arg= + */ + async removeKey(keyName: string): Promise { + /* ... */ + } +} +``` + +### 2.4 Publishing Path: Keep Pre-Signed Records for Client Publishes + +The current publishing flow has clients sign IPNS records locally and send pre-signed bytes to the API. This is correct for zero-knowledge -- the server never has the unencrypted IPNS private key. This does NOT change. + +**For client publishes:** The API receives pre-signed record bytes via `POST /ipns/publish`. The API: + +1. Upserts the `folder_ipns` DB record (unchanged -- this is the reliable CID source) +2. Broadcasts the pre-signed record to the DHT via Kubo's `/api/v0/routing/put` (replaces delegated-ipfs.dev) +3. Falls back to delegated-ipfs.dev only if Kubo routing put fails and the fallback is enabled + +**For TEE republishes:** The existing flow is preserved: the TEE decrypts the IPNS private key in hardware, signs the IPNS record inside the TEE boundary, and returns the **signed record bytes** (not the raw key) to the API. The API then broadcasts the pre-signed record to the DHT via Kubo's `/api/v0/routing/put`, replacing delegated-ipfs.dev as the broadcast target. + +The IPNS private key never leaves the TEE boundary. This maintains the security invariant that the API server never handles raw IPNS private keys. + +**Rejected alternative (Kubo key import):** An optimization was considered where the TEE returns raw Ed25519 private key bytes for temporary import into Kubo's keystore (`/api/v0/key/import` → `/api/v0/name/publish` → `/api/v0/key/rm`). This was rejected because it would export the private key from the trusted boundary, undermining the TEE security model even if the key is only transiently present in Kubo's memory. + +### 2.5 Kubo Configuration Changes + +Enable on the self-hosted Kubo node: + +```json +{ + "Ipns": { + "UsePubsub": true + }, + "Routing": { + "Type": "auto" + } +} +``` + +`Routing.Type: "auto"` (default) uses both DHT and any configured delegated routers. `Ipns.UsePubsub: true` enables near-instant resolution when both publisher and resolver subscribe to the same pubsub topic. This is especially useful for same-node operations (client publishes, then immediately resolves on the same Kubo instance). + +### 2.6 Impact on Recovery Tool + +The recovery tool (`apps/web/public/recovery.html`) currently resolves IPNS via `delegated-ipfs.dev` directly from the browser (no API needed -- the tool works offline from the CipherBox API). + +Updated resolution order for recovery: + +1. Try the CipherBox API's `/ipns/resolve` endpoint (DB-first + Kubo DHT) +2. Fall back to `delegated-ipfs.dev` only if the API is unreachable (true offline recovery) +3. The tool should accept a manual CID input as a last resort (user pastes CID from backup) + +### 2.7 Modified Files + +| File | Change | +| ----------------------------------------------- | ---------------------------------------------------------------- | +| `apps/api/src/ipns/kubo-ipns.client.ts` | NEW -- Kubo RPC client for IPNS operations | +| `apps/api/src/ipns/kubo-ipns.client.spec.ts` | NEW -- Unit tests | +| `apps/api/src/ipns/ipns.service.ts` | MODIFY `resolveRecord()` to DB-first with async DHT verification | +| `apps/api/src/ipns/delegated-routing.client.ts` | MODIFY -- Add config flag to disable, demote to fallback | +| `apps/api/src/ipns/ipns.module.ts` | MODIFY -- Register `KuboIpnsClient` | +| `apps/api/src/republish/republish.service.ts` | MODIFY -- Option to use Kubo native publish for TEE path | +| `apps/api/.env.example` | ADD `IPNS_DELEGATED_ROUTING_ENABLED=false` | +| `apps/web/public/recovery.html` | MODIFY -- Updated resolution order | + +--- + +## 3. Database Minimization: Migration Path + +### 3.1 Table-by-Table Analysis + +#### 3.1.1 `vaults.encryptedRootFolderKey` -- MIGRATE TO IPFS + +**Current state:** The `vaults` table stores `encryptedRootFolderKey` (ECIES-wrapped with user's publicKey) and `encryptedRootIpnsPrivateKey` (also ECIES-wrapped, but redundant since the IPNS key is HKDF-derivable). + +**Migration:** Move `encryptedRootFolderKey` into the IPFS blob pointed at by the root vault IPNS record. The blob currently contains only AES-GCM encrypted folder metadata. The new format prepends the ECIES-wrapped root folder key. + +**New blob format (version 2):** + +```text +Vault IPFS Blob v2: + Byte 0: version = 0x02 + Bytes 1-2: encryptedRootFolderKey length (uint16, big-endian) + Bytes 3..N: ECIES-encrypted rootFolderKey + Bytes N+1..: AES-GCM encrypted folder metadata (unchanged) + +Vault IPFS Blob v1 (current, no version byte): + All bytes: AES-GCM encrypted folder metadata +``` + +Detection: If the first byte is `0x02`, parse as v2. If the first bytes are a valid AES-GCM ciphertext (IV + ciphertext + tag), parse as v1. + +**Login flow change:** + +Before: + +```text +Login -> API GET /vault -> { encryptedRootFolderKey, rootIpnsName } + -> Derive IPNS key via HKDF + -> Resolve IPNS -> fetch blob -> decrypt metadata with rootFolderKey +``` + +After: + +```text +Login -> Derive IPNS key via HKDF (no API call needed for key material) + -> API GET /ipns/resolve?ipnsName= -> { cid } + -> API GET /ipfs/ -> blob v2 + -> Extract encryptedRootFolderKey from blob header + -> ECIES decrypt rootFolderKey with privateKey + -> Decrypt folder metadata with rootFolderKey +``` + +**Migration strategy (dual-write, version-aware read):** + +1. **Write path:** Client publishes root metadata in blob v2 format (encryptedRootFolderKey in header). ALSO continues sending encryptedRootFolderKey in the vault init call for backward compatibility with older clients and the current recovery tool. +2. **Read path:** Client checks first byte of blob. v2 -> extract key from blob. v1 or unrecognizable -> fall back to `GET /vault` for the key. +3. **Cutover criteria:** When all active clients support blob v2 reading, the `GET /vault` endpoint can stop returning `encryptedRootFolderKey`. The column is kept in the DB as disaster recovery fallback but is no longer the primary source. + +#### 3.1.2 `vaults.encryptedRootIpnsPrivateKey` -- ELIMINATE + +This field is redundant. The root IPNS private key is deterministically derivable from the user's secp256k1 private key via HKDF: + +```text +rootIpnsPrivateKey = HKDF(userPrivateKey, info="cipherbox-root-ipns") +``` + +The client already performs this derivation. The server-stored copy was a bootstrapping convenience from v0.1 that is no longer needed. Stop sending it in vault init. Keep the DB column for backward compatibility but mark it deprecated. + +#### 3.1.3 `folder_ipns` -- KEEP BUT REDUCE ROLE + +The `folder_ipns` table currently serves four purposes: + +1. **CID cache** -- Reliable fallback when IPNS DHT resolution fails +2. **Sequence number tracking** -- Optimistic concurrency control (409 Conflict detection) +3. **Encrypted IPNS key storage** -- For TEE enrollment +4. **Record type metadata** -- folder vs file distinction + +With reliable Kubo IPNS resolution (section 2): + +- Purpose 1 (CID cache) is still valuable as a fast path (<5ms vs seconds for DHT), but no longer critical as the sole reliable source +- Purpose 2 (sequence numbers) is essential and CANNOT move to IPFS. You need to know the current sequence BEFORE publishing. This is inherently a server-side coordination concern. +- Purpose 3 (encrypted IPNS key) is duplicated in `ipns_republish_schedule`. Can be deduplicated by having the republish schedule reference `folder_ipns` instead of storing its own copy. Minor cleanup. +- Purpose 4 (record type) is metadata, could be derived but not worth the effort to remove. + +**Recommendation:** Keep `folder_ipns` as a "publish coordination table." Rename its conceptual role in documentation. Do NOT attempt to eliminate it -- sequence number tracking is a hard requirement for conflict detection. + +#### 3.1.4 `shares`, `share_keys` -- FUTURE (CRDT INBOX RESEARCH ONLY) + +Per the CRDT-based IPNS inbox todo, this milestone performs research only. The tables stay as-is. Key findings: + +- G-Set CRDT solves concurrent share additions (append-only inbox) +- Write-access control via signed envelopes prevents inbox spam +- Revocation by key rotation (not by modifying inbox) aligns with existing lazy revocation pattern +- State size growth needs compaction strategy +- **Dependency:** Requires reliable IPNS resolution (section 2) before inbox-based discovery is viable +- **Scope for v1.1:** Document the CRDT approach as a design RFC. Do NOT implement. + +#### 3.1.5 `device_approvals` -- MIGRATE TO IPFS (POSSIBLE BUT NOT RECOMMENDED) + +Device approvals are short-lived (15-minute expiry), require real-time status updates (pending -> approved/denied), and involve server-mediated notification polling. IPNS polling at 30s intervals is too slow for MFA approval UX. Keep server-side. + +#### 3.1.6 `pinned_cids` -- KEEP (QUOTA ENFORCEMENT) + +The server must enforce storage quotas. If quota tracking moves to IPFS, a malicious client could lie about their storage usage. The server must independently track pinned CID sizes. + +### 3.2 Post-Migration Database Role + +After v1.1 migrations, the database serves these purposes: + +| Purpose | Tables | Why Server-Side | +| -------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------- | +| Identity | `users`, `auth_methods` | Server-mediated auth | +| Sessions | `refresh_tokens` | JWT lifecycle | +| Quota enforcement | `pinned_cids` | Server must prevent quota abuse | +| Publish coordination | `folder_ipns` | Sequence numbers for concurrency | +| TEE orchestration | `ipns_republish_schedule`, `tee_key_state`, `tee_key_rotation_log` | Server schedules TEE work | +| Sharing graph | `shares`, `share_keys`, `share_invites` | Until CRDT inbox (v1.2+) | +| MFA | `device_approvals` | Short-lived approval state | +| Vault metadata | `vaults` (reduced) | ownerPublicKey, rootIpnsName, encryptedRootFolderKey (permanent fallback) | + +**What changed:** `encryptedRootIpnsPrivateKey` is deprecated (HKDF-derivable, redundant). `encryptedRootFolderKey` is retained as a permanent DB fallback even after migration to vault blob v2 on IPFS -- the IPFS copy is the primary source for recovery tool independence, but the DB copy ensures login works even if IPNS resolution fails. The server's role shifts from key escrow to coordination relay, though it still holds one ECIES-wrapped key for resilience. + +--- + +## 4. BYO-IPFS Node Support + +### 4.1 Recommendation: Server-Relay Model with Provider Abstraction + +**Decision:** Server-relay (not client-direct) because: + +1. **Quota tracking:** Server must see upload sizes to enforce quota. Client-direct bypasses quota enforcement entirely. +2. **IPNS publish coordination:** Sequence numbers are tracked server-side in `folder_ipns`. Bypassing the API means no conflict detection. +3. **CORS/connectivity:** Kubo's RPC API has no CORS headers by default. Client-direct from the browser requires the user's node to be publicly accessible or configured with CORS -- unreliable for home nodes behind NAT. +4. **Credential safety:** Server-relay means user's IPFS node credentials (API keys, auth tokens) are stored server-side, but the data passing through is already AES-256-GCM encrypted ciphertext. The server cannot read content regardless of relay model. + +**Future exception:** Desktop clients (Tauri) can pin directly to a user's local Kubo node because there are no CORS restrictions and the desktop app is trusted. This is a v1.2+ enhancement. + +### 4.2 Provider Abstraction Extension + +Extend the existing `IpfsProvider` with a per-user factory: + +```typescript +// apps/api/src/ipfs/providers/ipfs-provider.interface.ts -- ADDITIONS +export interface IpfsProviderFactory { + /** Get the appropriate IPFS provider for a given user */ + getProvider(userId: string): Promise; + /** Get the default (CipherBox-managed) provider */ + getDefaultProvider(): IpfsProvider; +} + +export const IPFS_PROVIDER_FACTORY = 'IPFS_PROVIDER_FACTORY'; +``` + +The factory checks for per-user IPFS configuration. If none exists, it returns the default `LocalProvider`. If the user has a custom node configured, it returns a dual-pin provider that pins to both nodes. + +### 4.3 New Provider: UserCustomProvider + +```typescript +// apps/api/src/ipfs/providers/user-custom.provider.ts +export class UserCustomProvider implements IpfsProvider { + constructor( + private readonly endpoint: string, + private readonly authToken?: string, + private readonly providerType: 'kubo' | 'pinning-api' + ) {} + + async pinFile(data: Buffer): Promise<{ cid: string; size: number }> { + if (this.providerType === 'kubo') { + // Kubo RPC: POST /api/v0/add?pin=true&cid-version=1 + return this.pinViaKuboApi(data); + } else { + // IPFS Pinning Service API (spec: ipfs.github.io/pinning-services-api-spec) + // POST /pins with CID (requires content already available on IPFS network) + // For fresh uploads, use /api/v0/add equivalent or add+pin flow + return this.pinViaPinningApi(data); + } + } + + async unpinFile(cid: string): Promise { + /* provider-specific unpin */ + } + async getFile(cid: string): Promise { + /* provider-specific fetch */ + } +} +``` + +**Provider types supported:** + +| Provider Type | Protocol | Examples | Auth Method | +| ------------- | -------------------------- | ------------------------------ | ------------------------------ | +| `kubo` | Kubo RPC API (`/api/v0/*`) | Self-hosted Kubo, IPFS Desktop | None (localhost) or Basic Auth | +| `pinning-api` | IPFS Pinning Service API | Pinata, Filebase, web3.storage | Bearer token | + +### 4.4 Dual-Pin Provider + +The actual provider used for BYO-IPFS users wraps both the default and custom providers: + +```typescript +// apps/api/src/ipfs/providers/dual-pin.provider.ts +export class DualPinProvider implements IpfsProvider { + constructor( + private readonly defaultProvider: IpfsProvider, + private readonly customProvider: IpfsProvider + ) {} + + async pinFile(data: Buffer): Promise<{ cid: string; size: number }> { + // Always pin to default (CipherBox) node first -- this is the reliable source + const result = await this.defaultProvider.pinFile(data); + + // Best-effort pin to user's custom node + try { + await this.customProvider.pinFile(data); + } catch (error) { + // Log warning but do NOT fail the upload + logger.warn(`BYO-IPFS pin failed: ${error.message}`); + } + + return result; // Return CID from default node + } + + async getFile(cid: string): Promise { + // Try default node first, fall back to custom + try { + return await this.defaultProvider.getFile(cid); + } catch { + return await this.customProvider.getFile(cid); + } + } + + async unpinFile(cid: string): Promise { + // Only unpin from CipherBox's node (user manages their own retention) + await this.defaultProvider.unpinFile(cid); + } +} +``` + +### 4.5 User Settings Entity + +```sql +CREATE TABLE user_ipfs_config ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL UNIQUE REFERENCES users(id) ON DELETE CASCADE, + provider_type VARCHAR(20) NOT NULL DEFAULT 'default', -- 'default' | 'kubo' | 'pinning-api' + endpoint_url VARCHAR(500), + auth_token_encrypted BYTEA, -- encrypted with server-side key + created_at TIMESTAMP DEFAULT NOW(), + updated_at TIMESTAMP DEFAULT NOW() +); +``` + +**`auth_token_encrypted`:** Encrypted with a server-side symmetric key (from env var), NOT the user's key. The server needs to decrypt it to authenticate with the user's IPFS node. This is acceptable because the data being relayed is already AES-256-GCM encrypted ciphertext -- the server cannot read it regardless. + +### 4.6 API Endpoints + +```text +GET /vault/ipfs-config Get user's IPFS node configuration +PUT /vault/ipfs-config Set/update IPFS node configuration +DELETE /vault/ipfs-config Remove custom config (revert to default) +POST /vault/ipfs-config/test Test connectivity to user's IPFS node +``` + +The test endpoint attempts a small pin+unpin operation on the user's node to verify connectivity and auth. + +### 4.7 Web UI: Settings Page Extension + +Add an "IPFS Node" section to the existing settings page: + +```text +[IPFS Storage] + Provider: ( ) CipherBox Default ( ) Custom Kubo Node ( ) Pinning Service + Endpoint: [http://my-node:5001 ] + Auth Token: [optional, for pinning services ] + [Test Connection] [Save Configuration] + + Status: Connected -- last verified 2 minutes ago +``` + +### 4.8 IPNS and Conflict Detection Implications + +BYO-IPFS affects ONLY where encrypted data is pinned. It does NOT affect: + +- **IPNS publishing:** All publishes still go through the CipherBox API. The client signs IPNS records, the API broadcasts to the DHT. +- **Sequence numbers:** Still tracked in `folder_ipns`. Conflict detection is unchanged. +- **TEE republishing:** Unaffected -- TEE uses CipherBox's Kubo node. +- **IPNS resolution:** Unaffected -- IPNS resolves to a CID, and the CID is available from any node that has it pinned. + +If a future version allows publishing directly to the user's node (bypassing the API), conflict detection would need to move client-side. This is NOT in v1.1 scope. + +### 4.9 Modified/New Files + +| File | Change | +| -------------------------------------------------------------- | ---------------------------------------------------------- | +| `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` | ADD `IpfsProviderFactory` interface | +| `apps/api/src/ipfs/providers/user-custom.provider.ts` | NEW -- Custom Kubo/Pinning API provider | +| `apps/api/src/ipfs/providers/user-custom.provider.spec.ts` | NEW -- Unit tests | +| `apps/api/src/ipfs/providers/dual-pin.provider.ts` | NEW -- Dual-pin wrapper provider | +| `apps/api/src/ipfs/providers/dual-pin.provider.spec.ts` | NEW -- Unit tests | +| `apps/api/src/ipfs/providers/provider-factory.service.ts` | NEW -- Per-user provider resolution | +| `apps/api/src/ipfs/providers/provider-factory.service.spec.ts` | NEW -- Unit tests | +| `apps/api/src/ipfs/ipfs.module.ts` | MODIFY -- Register factory, user config repository | +| `apps/api/src/ipfs/ipfs.controller.ts` | MODIFY -- Use factory instead of direct provider injection | +| `apps/api/src/vault/entities/user-ipfs-config.entity.ts` | NEW -- User IPFS config entity | +| `apps/api/src/vault/dto/ipfs-config.dto.ts` | NEW -- IPFS config CRUD DTOs | +| `apps/api/src/vault/vault.controller.ts` | MODIFY -- Add IPFS config endpoints | +| `apps/api/src/migrations/XXXXXXXXX-AddUserIpfsConfig.ts` | NEW -- Create table migration | +| `apps/web/src/components/settings/IpfsNodeConfig.tsx` | NEW -- IPFS config UI | +| `apps/web/src/routes/SettingsPage.tsx` | MODIFY -- Add IPFS config section | + +--- + +## 5. Performance Instrumentation + +### 5.1 Existing Infrastructure + +The API already has Prometheus metrics via `prom-client`: + +- **Histogram:** `cipherbox_http_request_duration_seconds` with labels (method, route, status_code) and buckets [0.01 to 10s] +- **Counters:** file uploads/downloads/unpins, IPNS publishes/resolves, republish runs, auth logins +- **Gauges:** users total, files total, storage bytes, IPNS entries by type, republish schedule by status +- **Interceptor:** `HttpMetricsInterceptor` captures all HTTP request durations automatically +- **Endpoint:** `GET /metrics` exposes Prometheus-compatible output +- **Polling:** DB gauges refresh every 30 seconds + +### 5.2 New Server-Side Metrics + +Add domain-specific histograms for IPFS/IPNS latency tracking: + +```typescript +// IPNS resolution latency by source and outcome +readonly ipnsResolveDuration: client.Histogram; +// Labels: source (db|kubo_dht|delegated), result (hit|miss|error) +// Buckets: [0.005, 0.01, 0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10] + +// IPNS publish latency by target and outcome +readonly ipnsPublishDuration: client.Histogram; +// Labels: target (db|kubo_dht|delegated), result (success|error) +// Buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10, 30] + +// IPFS operation latency (pin/unpin/get) +readonly ipfsOperationDuration: client.Histogram; +// Labels: operation (pin|unpin|get), provider (default|byo), result (success|error) +// Buckets: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10] + +// TEE republish batch processing duration +readonly republishBatchDuration: client.Histogram; +// Labels: result (success|partial|error) +// Buckets: [0.5, 1, 2.5, 5, 10, 30, 60, 120] +``` + +### 5.3 Client-Side Performance Utility + +Add a lightweight timing utility for the web app: + +```typescript +// apps/web/src/lib/perf.ts +export function startTimer(label: string): () => number { + const start = performance.now(); + return () => { + const durationMs = performance.now() - start; + logger.debug(`[perf] ${label}: ${durationMs.toFixed(1)}ms`); + return durationMs; + }; +} +``` + +**Key operations to instrument:** + +| Operation | File | What to Measure | +| ----------------------------------- | ------------------------ | -------------------------------------- | +| IPNS resolve (API round trip) | `folder.service.ts` | API call latency | +| IPNS publish (API round trip) | `folder.service.ts` | API call + DB + DHT publish | +| File upload (encrypt + upload) | `upload.service.ts` | Full pipeline including encryption | +| File download (fetch + decrypt) | `download.service.ts` | Full pipeline including decryption | +| Folder metadata decrypt | `folder.service.ts` | AES-GCM decryption of metadata blob | +| AES-GCM encrypt throughput | `packages/crypto` | Encryption speed in MB/s | +| Full page load to interactive | `useAuth.ts` | Auth completion to first folder render | +| Folder navigation (click to render) | `useFolderNavigation.ts` | Navigate + resolve + decrypt + render | + +### 5.4 Baseline Collection Strategy + +1. **Instrument:** Add timing hooks with minimal overhead (no-op when not collecting) +2. **Collect baselines:** Run standardized workloads against staging environment +3. **Document:** Store baselines in `.planning/baselines/PERFORMANCE_BASELINES.md` with environment specs, dates, and methodology +4. **Automate (stretch goal):** Add a Playwright E2E test suite that measures key user journeys and asserts maximum latencies + +**Standardized workload scenarios:** + +| Scenario | Description | Target Metric | +| --------------------------------- | -------------------------------------------------- | ---------------------------- | +| Login flow | Auth -> vault init -> root resolve -> first render | Total wall time | +| Upload 1 MB | Encrypt -> API upload -> IPNS publish -> confirm | Total wall time | +| Upload 50 MB | Same as above, larger payload | Total wall time, memory peak | +| Download 1 MB | API fetch -> decrypt -> save | Total wall time | +| Navigate 3 levels deep | Resolve root -> subfolder -> subfolder | Total wall time | +| IPNS publish + resolve round trip | Publish -> immediately resolve same name | Latency | +| Search 100 files | Index lookup -> render results | Latency | +| Share a folder | Wrap key -> API call -> confirm | Total wall time | + +### 5.5 Modified/New Files + +| File | Change | +| ----------------------------------------------- | ---------------------------------------------- | +| `apps/api/src/metrics/metrics.service.ts` | ADD new histograms (4 new metrics) | +| `apps/api/src/ipns/ipns.service.ts` | ADD histogram observations in resolve/publish | +| `apps/api/src/ipfs/ipfs.controller.ts` | ADD histogram observations in pin/get/unpin | +| `apps/api/src/republish/republish.service.ts` | ADD histogram observation for batch processing | +| `apps/web/src/lib/perf.ts` | NEW -- Client-side performance timing utility | +| `apps/web/src/services/folder.service.ts` | ADD timing instrumentation | +| `apps/web/src/services/upload.service.ts` | ADD timing instrumentation | +| `apps/web/src/services/download.service.ts` | ADD timing instrumentation | +| `tests/e2e/tests/performance-baselines.spec.ts` | NEW -- Baseline collection E2E test (stretch) | + +--- + +## 6. Component Boundary Map + +### New Components + +```text +apps/api/src/ + ipns/ + kubo-ipns.client.ts NEW -- Kubo RPC client for IPNS + kubo-ipns.client.spec.ts NEW -- Unit tests + + ipfs/ + providers/ + user-custom.provider.ts NEW -- BYO-IPFS provider (Kubo + Pinning API) + user-custom.provider.spec.ts NEW -- Unit tests + dual-pin.provider.ts NEW -- Dual-pin wrapper + dual-pin.provider.spec.ts NEW -- Unit tests + provider-factory.service.ts NEW -- Per-user provider resolution + provider-factory.service.spec.ts NEW -- Unit tests + + vault/ + entities/ + user-ipfs-config.entity.ts NEW -- User IPFS node config entity + dto/ + ipfs-config.dto.ts NEW -- IPFS config CRUD DTOs + + migrations/ + XXXXXXXXX-AddUserIpfsConfig.ts NEW -- user_ipfs_config table + +apps/web/src/ + lib/ + perf.ts NEW -- Performance timing utility + + components/ + settings/ + IpfsNodeConfig.tsx NEW -- IPFS node config UI +``` + +### Modified Components + +```text +apps/api/src/ + ipns/ + ipns.service.ts MODIFY -- DB-first resolution, async DHT verify + ipns.module.ts MODIFY -- Register KuboIpnsClient + delegated-routing.client.ts MODIFY -- Add disable flag, demote to fallback + + ipfs/ + ipfs.module.ts MODIFY -- Use provider factory pattern + ipfs.controller.ts MODIFY -- Use factory, add perf instrumentation + + vault/ + vault.service.ts MODIFY -- Handle vault blob v2 format + vault.controller.ts MODIFY -- Add IPFS config endpoints + + republish/ + republish.service.ts MODIFY -- Optional Kubo native publish for TEE + + metrics/ + metrics.service.ts MODIFY -- Add 4 new histograms + +apps/web/src/ + services/ + folder.service.ts MODIFY -- Vault blob v2 reading, perf timers + upload.service.ts MODIFY -- Perf instrumentation + download.service.ts MODIFY -- Perf instrumentation + + routes/ + SettingsPage.tsx MODIFY -- Add IPFS node config section + + public/ + recovery.html MODIFY -- Vault blob v2, updated IPNS resolution + +packages/crypto/src/ + vault/ + types.ts MODIFY -- Add blob v2 type definitions + blob.ts NEW -- Blob v2 serialization/deserialization +``` + +### Inter-Component Communication + +```text +IpfsController --> ProviderFactory --> LocalProvider (default) + --> DualPinProvider --> LocalProvider (always) + --> UserCustomProvider (best-effort) + +IpnsController --> IpnsService --> folder_ipns DB (primary, fast path) + --> KuboIpnsClient (async DHT verification) + --> DelegatedRoutingClient (disabled-by-default fallback) + +RepublishService --> TeeService (signing) + --> KuboIpnsClient (Kubo native publish, preferred) + --> DelegatedRoutingClient (fallback if Kubo fails) + +MetricsService <-- IpnsService (resolve/publish duration) + <-- IpfsController (operation duration) + <-- RepublishService (batch duration) + <-- ProviderFactory (BYO-IPFS operation duration) +``` + +--- + +## 7. Data Flow Changes + +### 7.1 IPNS Resolution (Changed) + +**Before:** + +```text +Client -> API /ipns/resolve + -> DelegatedRoutingClient.resolve() [10s timeout, 3 retries] + -> Parse IPNS record bytes + -> Compare with DB-cached CID (folder_ipns) + -> Return highest sequence number result +``` + +**After:** + +```text +Client -> API /ipns/resolve + -> DB query folder_ipns WHERE ipnsName = ? [<5ms] + -> Return DB result immediately + -> Fire-and-forget: KuboIpnsClient.resolve() [5s DHT timeout] + -> If DHT has higher sequence number, update folder_ipns + -> If DHT fails, no-op +``` + +### 7.2 IPNS Publishing (Changed for TEE Path) + +**Before (TEE republish):** + +```text +RepublishService -> TeeService.republish() -> signed record bytes + -> DelegatedRoutingClient.publish() -> delegated-ipfs.dev +``` + +**After (TEE republish, preferred path):** + +```text +RepublishService -> TeeService.republish() -> signed record bytes + -> KuboIpnsClient.importKey() -> Kubo keystore + -> KuboIpnsClient.publish() -> Kubo DHT native + -> KuboIpnsClient.removeKey() -> cleanup + -> Fallback: DelegatedRoutingClient.publish() if Kubo fails +``` + +### 7.3 Vault Access on Login (Changed) + +**Before:** + +```text +Client -> API GET /vault -> { encryptedRootFolderKey, rootIpnsName, teeKeys } + -> Derive IPNS key via HKDF + -> API GET /ipns/resolve -> { cid } + -> API GET /ipfs/ -> blob v1 (encrypted metadata only) + -> Decrypt metadata with rootFolderKey from vault response +``` + +**After (v1.1, with blob v2):** + +```text +Client -> API GET /vault -> { rootIpnsName, teeKeys, ownerPublicKey } + -> Derive IPNS key via HKDF + -> API GET /ipns/resolve -> { cid } + -> API GET /ipfs/ -> blob v2 + -> Parse blob: extract encryptedRootFolderKey header + -> ECIES decrypt rootFolderKey with privateKey + -> Decrypt metadata with rootFolderKey + -> Fallback: if blob is v1, GET /vault.encryptedRootFolderKey +``` + +### 7.4 BYO-IPFS Upload (New) + +```text +Client -> encrypt file -> API POST /ipfs/upload { file } + -> ProviderFactory.getProvider(userId) + -> If user has custom config: + -> DualPinProvider.pinFile(data) + -> LocalProvider.pinFile(data) // CipherBox node (always, reliable) + -> UserCustomProvider.pinFile(data) // User node (best-effort, logged) + -> If no custom config: + -> LocalProvider.pinFile(data) // CipherBox node only + -> VaultService.recordPin(userId, cid, size) // Quota tracking (unchanged) + -> return { cid, size } +``` + +### 7.5 BYO-IPFS Download (New Path) + +```text +Client -> API GET /ipfs/:cid + -> ProviderFactory.getProvider(userId) + -> If user has custom config: + -> DualPinProvider.getFile(cid) + -> Try LocalProvider.getFile(cid) // CipherBox node first + -> If not found: UserCustomProvider.getFile(cid) // Fall back to user node + -> If no custom config: + -> LocalProvider.getFile(cid) + -> return file buffer +``` + +--- + +## 8. Suggested Build Order + +### Phase 1: Performance Instrumentation + +**Why first:** Zero risk to existing functionality. Purely additive. Establishes baselines BEFORE making any architectural changes. Without baselines, we cannot measure whether subsequent phases improve or regress performance. + +**Scope:** + +- Add 4 new Prometheus histograms to `MetricsService` +- Instrument `IpnsService.resolveRecord()` and `publishRecord()` with timing +- Instrument `IpfsController` upload/download/unpin with timing +- Instrument `RepublishService.processRepublishBatch()` with timing +- Create client-side `perf.ts` utility +- Instrument key web app operations (folder navigate, upload, download) +- Run standardized baseline collection against staging +- Document baselines in `.planning/baselines/` + +**Dependencies:** None. Uses existing `prom-client` and `MetricsService` infrastructure. +**Risk:** LOW. Additive instrumentation only. No behavioral changes. +**Estimated scope:** Small. ~10 modified files, no new entities or migrations. + +### Phase 2: IPNS Resolution Improvement + +**Why second:** Fixes the most critical reliability issue (delegated-ipfs.dev dependency). Required before Phase 3 because moving rootFolderKey to IPFS makes IPNS resolution a login-critical path. Baselines from Phase 1 enable before/after comparison. + +**Scope:** + +- Implement `KuboIpnsClient` for native Kubo IPNS resolution +- Refactor `IpnsService.resolveRecord()` to DB-first with async DHT verification +- Make `DelegatedRoutingClient` optional with config flag (`IPNS_DELEGATED_ROUTING_ENABLED`) +- Configure Kubo with `Ipns.UsePubsub: true` +- Optionally update `RepublishService` to use Kubo native publish +- Update recovery tool IPNS resolution fallback chain +- Measure resolution latency against Phase 1 baselines + +**Dependencies:** Phase 1 (for before/after measurement). +**Risk:** MEDIUM. Changing the resolution path could cause resolution failures if Kubo DHT is slower than expected. Mitigated by: (1) DB-first strategy means the fast path is always local, (2) delegated routing remains as a disabled-by-default fallback, (3) existing retry logic in the delegated client. +**Estimated scope:** Medium. 1 new file, ~5 modified files, Kubo config change. + +### Phase 3: Database Minimization (rootFolderKey to IPFS) + +**Why third:** Requires reliable IPNS resolution (Phase 2). The vault blob v2 format is a breaking metadata change that needs careful migration across web, desktop, and recovery tool. + +**Scope:** + +- Define vault blob v2 format in `packages/crypto/src/vault/` +- Implement blob v2 serialization/deserialization +- Implement dual-write in web client (blob v2 on publish, DB on vault init) +- Implement version-aware blob reading (v2 -> extract key, v1 -> fall back to DB) +- Update vault init flow to make encryptedRootFolderKey optional +- Mark encryptedRootIpnsPrivateKey as deprecated +- Update recovery tool for blob v2 format +- Update desktop client blob parsing +- Migration testing: new user, existing user upgrade, recovery +- Update `docs/METADATA_SCHEMAS.md` per evolution protocol + +**Dependencies:** Phase 2 (IPNS must be reliable before making it login-critical). +**Risk:** MEDIUM-HIGH. Metadata format change affects all clients. Mitigated by: (1) dual-write ensures backward compatibility, (2) version detection allows graceful fallback, (3) DB column is never dropped -- kept as disaster recovery. +**Estimated scope:** Large. New type definitions, serialization code, changes in 3 clients (web, desktop, recovery). + +### Phase 4: BYO-IPFS Node Support + +**Why last:** Largest surface area (new entity, new providers, new UI, new API endpoints). Benefits from stable IPNS resolution (Phase 2) and reduced DB dependency (Phase 3). Independent in design but benefits from the improved infrastructure. + +**Scope:** + +- Create `user_ipfs_config` entity and migration +- Implement `UserCustomProvider` with Kubo RPC and Pinning Service API support +- Implement `DualPinProvider` wrapper +- Implement `ProviderFactory` for per-user provider resolution +- Refactor `IpfsController` to use factory +- Add IPFS config CRUD endpoints to vault controller +- Add connection test endpoint +- Build IPFS node configuration UI in settings page +- Implement dual-pin strategy (CipherBox node always + user node best-effort) +- Add BYO-IPFS specific Prometheus metrics +- E2E test: configure custom node, upload, verify pinned + +**Dependencies:** Phase 2 (reliable IPNS) and Phase 3 (reduced DB) are not strict blockers but provide a better foundation. +**Risk:** MEDIUM. New provider implementations need thorough testing against real IPFS nodes and pinning services. Connectivity to arbitrary user nodes introduces unpredictable failure modes. Mitigated by: (1) dual-pin always keeps a copy on CipherBox node, (2) best-effort approach for user node, (3) connection test endpoint validates before saving config. +**Estimated scope:** Large. New entity, migration, 4+ new files, UI component, multiple API endpoints. + +### Build Order Summary + +```text +Phase 1: Performance Instrumentation + [No dependencies, zero-risk baseline establishment] + | + v +Phase 2: IPNS Resolution Improvement + [Eliminates delegated-ipfs.dev, enables Phase 3] + | + v +Phase 3: Database Minimization + [Moves rootFolderKey to IPFS, requires reliable IPNS from Phase 2] + | + v +Phase 4: BYO-IPFS Node Support + [New capability, benefits from improved infrastructure] +``` + +**Critical dependency:** Phase 2 MUST precede Phase 3. Moving rootFolderKey to IPFS without reliable IPNS resolution means users cannot log in if Kubo DHT is slow. + +**Parallel opportunity:** Phase 4's design and provider implementation can be developed in parallel with Phase 3's blob format work, but integration testing should wait for Phase 3 completion. + +--- + +## 9. Sources + +### HIGH Confidence (Official Documentation, Verified) + +- [Kubo RPC API v0 Reference](https://docs.ipfs.tech/reference/kubo/rpc/) -- `/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import` endpoints verified against v0.40.0 +- [Kubo Configuration Reference](https://github.com/ipfs/kubo/blob/master/docs/config.md) -- `Ipns.UsePubsub`, `Routing.Type` configuration +- [IPFS Pinning Service API Spec](https://ipfs.github.io/pinning-services-api-spec/) -- Vendor-agnostic pinning API standard (OpenAPI spec) +- [IPNS Concepts](https://docs.ipfs.tech/concepts/ipns/) -- IPNS TTL default (5 minutes), PubSub resolution +- [prom-client GitHub](https://github.com/siimon/prom-client) -- Histogram, Counter, Gauge APIs + +### MEDIUM Confidence (Multiple Sources Agreeing) + +- [Kubo v0.38+ Release Notes](https://github.com/ipfs/kubo/releases) -- Sweep provider with 97% fewer DHT lookups, IPNS TTL changes +- [Kubo Issue #10484: Download & Upload IPNS Records](https://github.com/ipfs/kubo/issues/10484) -- Confirms `/api/v0/routing/put` for pre-signed records +- [Kubo Issue #8542: Publish IPNS with Signature](https://github.com/ipfs/kubo/issues/8542) -- Confirms key import workflow +- [Kubo Delegated Routing Docs](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md) -- `put-ipns` and `get-ipns` routing methods +- [IP Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- DHT improvements context + +### Codebase Analysis (Direct Code Review) + +- `apps/api/src/ipns/ipns.service.ts` -- Current resolution logic, DB fallback (resolveRecord lines 290-350) +- `apps/api/src/ipns/delegated-routing.client.ts` -- Current publish/resolve implementation (3 retries, 10s timeout) +- `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` -- Existing 3-method provider interface +- `apps/api/src/ipfs/providers/local.provider.ts` -- Current Kubo provider (pin/unpin/get via RPC API) +- `apps/api/src/ipfs/ipfs.module.ts` -- Current DI config (env var driven, single provider) +- `apps/api/src/ipfs/ipfs.controller.ts` -- Current upload/download/unpin with metrics +- `apps/api/src/vault/vault.service.ts` -- Current vault init with encryptedRootFolderKey +- `apps/api/src/vault/entities/vault.entity.ts` -- Current vault schema (6 columns) +- `apps/api/src/metrics/metrics.service.ts` -- Existing Prometheus infrastructure (gauges, counters, histogram) +- `apps/api/src/metrics/http-metrics.interceptor.ts` -- Existing HTTP duration tracking +- `apps/api/src/republish/republish.service.ts` -- Current TEE republish via delegated routing +- `apps/api/src/ipns/entities/folder-ipns.entity.ts` -- CID cache + sequence tracking +- `apps/api/src/republish/republish-schedule.entity.ts` -- TEE scheduling state +- All 13 database entities analyzed for migration feasibility diff --git a/.planning/research/FEATURES.md b/.planning/research/_v1.1-archive/FEATURES.md similarity index 100% rename from .planning/research/FEATURES.md rename to .planning/research/_v1.1-archive/FEATURES.md diff --git a/.planning/research/_v1.1-archive/PITFALLS.md b/.planning/research/_v1.1-archive/PITFALLS.md new file mode 100644 index 0000000000..0c3adf55b2 --- /dev/null +++ b/.planning/research/_v1.1-archive/PITFALLS.md @@ -0,0 +1,346 @@ +# Pitfalls Research + +**Domain:** IPFS infrastructure improvements -- replacing delegated routing, migrating DB state to IPFS/IPNS, BYO-IPFS node support, and performance baselines for an existing zero-knowledge encrypted storage app +**Researched:** 2026-03-07 +**Confidence:** HIGH for routing/migration pitfalls (verified against codebase + IPFS docs), MEDIUM for BYO-IPFS (fewer production precedents), MEDIUM for instrumentation (well-understood domain but CipherBox-specific interactions need validation) + +**Context:** CipherBox v1.0 shipped 2026-03-05 with 423K lines of TypeScript + Rust across 698 source files. IPNS is already deeply integrated: the `folder_ipns` table tracks per-folder and per-file IPNS records with sequence numbers for optimistic concurrency, the `ipns_republish_schedule` table drives TEE republishing every 6 hours, and `delegated-ipfs.dev` is the sole external routing provider (known unreliable, with DB-cached CID fallback). The database stores auth, vault keys, shares, device approvals, pinned CIDs for quota tracking, and all IPNS state. The recovery tool (`recovery.html`) directly resolves IPNS via `delegated-ipfs.dev` without any API intermediary. + +--- + +## Critical Pitfalls + +Mistakes that cause data loss, break existing functionality, or require rearchitecting completed features. + +--- + +### Pitfall 1: Sequence Number Divergence When Switching Routing Providers + +**What goes wrong:** +The system currently has two sources of truth for IPNS sequence numbers: the `folder_ipns` PostgreSQL table and the DHT/delegated routing network. When replacing `delegated-ipfs.dev` with a self-hosted Someguy instance or Kubo DHT, the new routing endpoint may return different sequence numbers than the DB cache for a transition period. The `resolveRecord()` method in `ipns.service.ts` (line 290-350) already compares DB and network sequence numbers, taking the higher one. But during the switch, the new provider may have _no_ records (it has not seen previous publishes), causing resolution to return null from the network while the DB has valid data. If the resolution logic treats "null from network" differently than "null from delegated-ipfs.dev" -- or if the new provider starts returning stale records it picked up from DHT propagation with lower sequence numbers -- the system could serve outdated metadata, causing the client to see old folder contents or triggering false conflict detection (409 Conflict). + +The TEE republishing service (`republish.service.ts`) further complicates this: it publishes signed records to delegated routing and then syncs the sequence number back to `folder_ipns`. If the old and new routing providers are both receiving publishes during a transition window, the sequence number in each may diverge independently. + +**Why it happens:** + +- Developers assume a clean cutover is possible -- swap one URL for another +- IPNS records propagate through the DHT with eventual consistency; a new Someguy instance must discover existing records through DHT lookups, which takes time +- The TEE republisher runs on a 6-hour cycle and may publish to the old provider after the API has switched to the new one, creating a split-publish window + +**How to avoid:** + +1. Run the new routing provider in read-parallel mode first: configure it alongside `delegated-ipfs.dev` and compare results for at least 48 hours (one full DHT record expiry cycle) before cutting writes over +2. Seed the new provider by having the TEE republisher publish to BOTH providers during the transition. The `publishSignedRecord()` method at line 200-204 sends to one delegated routing client; extend this to dual-write +3. Only cut over reads after the new provider consistently returns sequence numbers >= the DB cache for a representative sample of IPNS names +4. Keep DB cache as the authoritative fallback throughout -- this already works and should remain the primary resolution path + +**Warning signs:** + +- IPNS resolve latency spikes or increased null-from-network responses in Prometheus metrics (`cipherbox_ipns_resolves_total` by `source` label) +- 409 Conflict errors on publish that were not present before the switch +- Recovery tool fails to resolve IPNS records (it bypasses the API and hits the routing provider directly) + +**Phase to address:** IPNS Reliability phase (first phase of the milestone) + +--- + +### Pitfall 2: Moving rootFolderKey to IPFS Creates a Hard Dependency on IPNS for Login + +**What goes wrong:** +Today, `rootFolderKey` is stored as ECIES-wrapped bytes in the `vaults` table (`encrypted_root_folder_key` column in `vault.entity.ts`). The login flow retrieves it via a direct PostgreSQL query -- fast, reliable, always available. The proposal to move it to an IPFS blob pointed at by the root vault IPNS record means that login now requires: (1) HKDF derivation of IPNS key, (2) IPNS resolution to get the CID, (3) IPFS fetch of the blob, (4) ECIES unwrapping. Steps 2 and 3 are network operations against infrastructure that has documented reliability issues. + +If IPNS resolution fails (which it does -- the entire motivation for this milestone), the user cannot log in because they cannot get their root folder key. The DB-cached CID fallback helps, but it means the database still stores CIDs pointing to encrypted root folder key blobs -- you have not actually eliminated the DB dependency, you have just moved what the DB stores from "ECIES-wrapped key bytes" to "CID of IPFS blob containing ECIES-wrapped key bytes." The net reduction in server-side state is zero unless you also eliminate the CID cache, at which point login depends entirely on IPNS reliability. + +**Why it happens:** + +- The goal of "minimize database to auth-only" is laudable but creates a conflict with the reliability requirement +- The rootFolderKey is unique: unlike IPNS private keys which are HKDF-derivable, the root folder key is a random AES-256 key. It literally cannot be rederived. Losing access to it means losing the entire vault. +- Developers see the encryption key as "just data that should live on IPFS" without recognizing it is the single most critical piece of data in the entire system + +**How to avoid:** + +1. Keep `encrypted_root_folder_key` in the database as the primary source. Optionally mirror it to IPFS for recovery tool independence, but the DB copy is canonical for login. +2. If the goal is recovery-tool independence from the server, embed the wrapped key in the IPFS vault blob AND keep the DB copy. The recovery tool reads from IPFS; the normal login reads from the DB. Both paths work. +3. Never make the sole access path for root folder key depend on IPNS resolution. This is a "both, not either" situation. +4. Per `METADATA_EVOLUTION_PROTOCOL.md`, this is a breaking change to the root vault blob format. Version the blob format so old clients can still decrypt old-format blobs during migration. + +**Warning signs:** + +- Login latency increases from <100ms to multi-second (IPNS resolve is median 11s on DHT per ProbeLab measurements) +- Login failure rate correlates with IPNS availability instead of being independent +- Recovery tool works but normal login fails (or vice versa) + +**Phase to address:** Database Minimization phase -- but this pitfall argues for NOT moving rootFolderKey off the DB at all, or at minimum maintaining a dual-write pattern + +--- + +### Pitfall 3: Migrating Shares to IPFS Breaks Recipient-Initiated Discovery + +**What goes wrong:** +The `shares` table enables recipient-initiated share discovery: a recipient calls `GET /shares/received` and the API queries `WHERE recipient_id = :userId`. If shares migrate to IPFS/IPNS (e.g., each user has an "inbox" IPNS record listing shares offered to them), the recipient must now know _where_ to look. In the current model, the server knows; in the IPFS model, either: + +(a) The sharer publishes to the recipient's inbox IPNS record -- but the sharer does not hold the recipient's IPNS private key and cannot sign the record, so this is architecturally impossible without a new key-sharing protocol. + +(b) The sharer publishes to their own IPNS record and the recipient polls all potential sharers -- but the recipient does not know who might share with them. + +(c) A server-side index maps recipients to share IPNS records -- but this puts the discovery mechanism back in the database, negating the migration. + +Additionally, the `share_keys` table stores per-file and per-subfolder ECIES-wrapped keys. Each key is specific to a recipient's public key. These cannot simply be "moved to IPFS" because they reference database UUIDs (`share_id`, `item_id`) for relational joins. The IPFS blob would need to embed all relationship data that currently lives in foreign keys and indices. + +**Why it happens:** + +- The mental model "move everything to IPFS" does not account for relational operations like indexed queries on foreign keys +- Share discovery is fundamentally a server-side operation in a system where users cannot enumerate other users' IPNS records +- The existing share-revocation flow uses soft-delete with `revoked_at` timestamp and lazy key rotation -- this stateful lifecycle is hard to model in immutable IPFS blobs + +**How to avoid:** + +1. Accept that shares, share_keys, and share_invites are server-side state that SHOULD stay in the database. They are access-control metadata, not user content. The server already sees `sharer_id`, `recipient_id`, and `ipns_name` in plaintext -- moving them to IPFS does not improve privacy. +2. If serverless share discovery is a goal (for BYO-IPFS or full decentralization), research CRDT-based IPNS inbox patterns BEFORE attempting migration. The todo at `2026-02-22-crdt-ipns-inbox-sharing.md` correctly flags this as research-only for this milestone. +3. Categorize DB tables into "can migrate" vs "must stay": + - **Can migrate:** `folder_ipns` (IPNS tracking), `pinned_cids` (quota tracking if BYO replaces it), vault keys (if dual-written) + - **Must stay:** `shares`, `share_keys`, `share_invites` (relational discovery), `device_approvals` (cross-device MFA), `users`, `auth_methods`, `refresh_tokens` (auth) + +**Warning signs:** + +- Design documents propose "share IPNS records" without specifying how recipients discover them +- The words "poll all sharers" appear in a design doc (quadratic complexity) +- Share acceptance latency goes from instant (DB query) to seconds (IPNS resolution) + +**Phase to address:** Database Minimization phase -- but the actual prevention is to scope "minimize database" more narrowly than "move everything off" + +--- + +### Pitfall 4: BYO-IPFS Bypasses Server-Side Optimistic Concurrency + +**What goes wrong:** +CipherBox's conflict detection works because ALL IPNS publishes go through the API, which checks `expectedSequenceNumber` against the `folder_ipns` table before accepting a publish (lines 177-189 of `ipns.service.ts`). When a user configures a BYO-IPFS node and publishes IPNS records directly (client-to-node without API relay), the server never sees the publish and cannot check sequence numbers. Two devices could publish conflicting metadata for the same folder, and the DHT will keep whichever has the higher sequence number -- but neither device knows about the conflict. + +Worse: the TEE republishing service tracks sequence numbers in `ipns_republish_schedule`. If a BYO user publishes directly to their node, the TEE's sequence number becomes stale. The next TEE republish will create a record with a LOWER sequence number than what the user published, causing the DHT to reject it (or worse, a race where different DHT nodes hold different records). The user's IPNS records silently stop being republished. + +**Why it happens:** + +- BYO-IPFS is designed for user sovereignty, but the entire concurrency model assumes server mediation +- The provider interface (`IpfsProvider`) only abstracts pin/unpin/fetch -- it does not abstract IPNS publishing, which is handled separately through `DelegatedRoutingClient` +- Client-direct upload to a user's Kubo node is the simplest implementation, but it bypasses every server-side check + +**How to avoid:** + +1. Even with BYO-IPFS, require IPNS publishes to go through the API. The server does not need to contact the user's IPFS node for IPNS -- it just needs to see the publish request for concurrency checking and TEE enrollment. +2. Separate the concerns: BYO-IPFS is about WHERE data is pinned, not about HOW metadata is published. Pin encrypted blobs to the user's node; publish IPNS records through the API. +3. If client-direct IPNS publishing is a requirement (full decentralization), implement client-side conflict detection: before publishing, resolve the current IPNS record, compare sequence numbers, and abort on mismatch. This is weaker than server-side checks (no atomicity guarantee) but better than nothing. +4. Add a "last known sequence number" field to the BYO-IPFS configuration so the client can detect when the TEE's sequence diverges from the user's actual sequence. + +**Warning signs:** + +- BYO users report "stale" folder contents on other devices +- TEE republish success rate drops for BYO users specifically +- Sequence numbers in `ipns_republish_schedule` are lower than what IPNS resolves to + +**Phase to address:** BYO-IPFS phase -- design decision required before implementation begins + +--- + +### Pitfall 5: DHT Record Expiry During Routing Provider Migration + +**What goes wrong:** +IPNS records on the DHT expire after 48 hours regardless of the validity field in the record. Kubo's default republish period is 4 hours, and CipherBox's TEE republishes every 6 hours. During a routing provider migration, if there is a gap where neither the old provider nor the new provider successfully publishes records, the 48-hour expiry clock keeps ticking. If the gap exceeds 48 hours (e.g., a weekend deployment where the new provider is misconfigured), ALL IPNS records for ALL users expire from the DHT simultaneously. Recovery requires republishing every single record -- but the TEE processes records in batches of 100 with a 6-hour cycle, so catching up on thousands of records takes many cycles. + +The DB-cached CID fallback saves resolution, but only for users going through the API. The recovery tool (`recovery.html`) and any future BYO-IPFS clients resolving directly from the DHT will get nothing. + +**Why it happens:** + +- The 48-hour DHT expiry is not widely known -- developers assume IPNS records persist until explicitly replaced +- The TEE republish cycle (6 hours) provides comfortable margin (48/6 = 8 missed cycles before expiry), creating false confidence +- Migration testing often happens on small test datasets where a full republish cycle completes quickly, masking the problem at scale + +**How to avoid:** + +1. Never take the old routing provider offline until the new one has successfully processed at least one full republish cycle for ALL enrolled records +2. Monitor the `cipherbox_republish_schedule_total` gauge by status during migration -- any spike in `retrying` or `stale` status indicates the new provider is not accepting publishes +3. Before migration, calculate the worst-case catch-up time: `(total enrolled records / BATCH_SIZE) * republish_interval = (N / 100) * 6 hours`. For 1000 records, that is 60 hours to fully catch up from scratch -- already past the 48-hour expiry +4. Consider a one-time "emergency republish" endpoint that processes all records immediately (bypassing the 6-hour schedule) after a provider switch +5. Reduce BATCH_SIZE temporarily during migration to process more records per cycle, or run catch-up cycles more frequently + +**Warning signs:** + +- `ipns_republish_schedule` entries accumulating in `retrying` or `stale` status +- IPNS resolution returning null for records that were previously resolvable +- Recovery tool stops working for all users simultaneously + +**Phase to address:** IPNS Reliability phase -- must be addressed in the migration runbook + +--- + +### Pitfall 6: Quota Tracking Becomes Unenforceable with BYO-IPFS + +**What goes wrong:** +CipherBox tracks storage quota via the `pinned_cids` table: every upload goes through `IpfsController`, which calls `pinFile()` on the server-managed Kubo node and records the CID + size in `pinned_cids`. The user's total is summed against a 500 MiB limit. With BYO-IPFS, if uploads go directly to the user's node (client-direct), the server never sees the upload and cannot record the CID. The `pinned_cids` table becomes incomplete. Quota enforcement is either (a) impossible, (b) advisory only, or (c) requires the client to self-report, which is trivially forgeable. + +The deeper issue: quota tracking exists to protect the CipherBox operator's IPFS node from unbounded storage consumption. If the user is pinning to their own node, the CipherBox operator has no storage cost and does not need to enforce quota. But the system currently conflates "enforce quota" with "track what the user has stored" -- and that tracking is also used for the recycle bin's CID unpinning (30-day retention + cleanup). + +**Why it happens:** + +- The existing quota model assumes all storage flows through the server +- Client-direct upload is the most obvious BYO architecture but breaks server-side bookkeeping +- Developers may add BYO pinning without considering that quota, recycle bin, and orphan cleanup all depend on the `pinned_cids` table + +**How to avoid:** + +1. For BYO-IPFS users, skip server-side quota enforcement entirely. The user manages their own node's capacity. +2. Still require BYO uploads to be reported to the API (POST with CID + size after client-side pin succeeds). This maintains the `pinned_cids` ledger for features that depend on it (recycle bin, orphan detection) without requiring the server to handle the actual data. +3. Mark `pinned_cids` entries with a `provider` column ("server" | "byo") so the system knows which CIDs it can directly unpin vs. which require the client to unpin. +4. Accept that unpinning on a BYO node requires client cooperation -- the server cannot reach into the user's node. Document this limitation clearly. + +**Warning signs:** + +- BYO users show 0 bytes used in quota display despite having files +- Recycle bin emptying fails silently for BYO-pinned CIDs (unpin calls go to the wrong node) +- Orphan CID detection reports all BYO CIDs as orphans + +**Phase to address:** BYO-IPFS phase + +--- + +## Technical Debt Patterns + +Shortcuts that seem reasonable but create long-term problems. + +| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable | +| -------------------------------------------------------------------- | --------------------------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Dual-write to old and new routing provider "forever" | Safe migration | Double the publish latency, double the failure surface, code complexity of maintaining two routing clients | Only during migration transition; set a hard deadline to remove the old provider within 2 weeks of cutover | +| Skip IPNS signature verification on DB-cached resolutions | Faster resolve, simpler code | DB-cached CIDs are not cryptographically verified -- a compromised DB could serve malicious CIDs | Acceptable only if the DB is trusted infrastructure. The web client at `ipns.service.ts:160-169` already verifies signatures from network resolves but skips it for DB-fallback results | +| BYO users self-report CID sizes for quota | Simple client-direct architecture | Users can lie about sizes; quota is advisory only | Acceptable for tech demonstrator. For production, require server verification of at least one byte-range to confirm size | +| Use DB-cached CID as primary resolution (skip IPNS network entirely) | Instant resolve, no DHT latency | DB becomes single point of truth for metadata location; IPNS becomes decorative | Acceptable as interim step. The current code already prefers DB when it has a newer sequence number. Making this explicit reduces complexity | +| Hardcode Someguy URL instead of making it configurable | Faster initial implementation | Cannot switch providers without code change and redeploy | Never -- use `DELEGATED_ROUTING_URL` env var (already exists in `delegated-routing.client.ts:22-25`). Zero cost to keep it configurable | +| Measure performance baselines during staging only | No production impact | Staging has different latency, load, and network characteristics than production | Never for baselines intended to represent production. Staging baselines are useful only for relative comparison (before/after a change) | + +--- + +## Integration Gotchas + +Common mistakes when connecting to external services or changing existing integrations. + +| Integration | Common Mistake | Correct Approach | +| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Someguy (self-hosted delegated routing) | Deploying Someguy without DHT bootstrap peers, resulting in an isolated node that cannot resolve any existing IPNS records | Configure Someguy with the Amino DHT bootstrap peers. Run it alongside Kubo so it benefits from Kubo's established DHT connections. Verify resolution of known IPNS names before cutting over | +| Kubo DHT direct (Routing.Type=auto) | Assuming the existing Kubo node already participates in DHT for IPNS. CipherBox's Kubo may be configured with DHT disabled (using only delegated routing) | Check `Routing.Type` in Kubo config. If it is "none" or "custom" with only delegated HTTP routers, IPNS DHT publishing/resolution is not happening through Kubo itself | +| IPFS Pinning Service API (for BYO) | Treating the Pinning Service API as equivalent to Kubo's `/api/v0/add`. The Pinning Service API is async -- `POST /pins` returns a `requestid` and pinning happens in the background | Poll pin status before assuming content is available. The CID may not be retrievable from the BYO node until pinning completes. Upload UX must reflect this ("pinning..." not "uploaded") | +| Recovery tool (`recovery.html`) | Changing the API's routing provider but forgetting the recovery tool resolves IPNS directly at line 363. The recovery tool is a standalone HTML file with no build system -- it does not use the API client | Update the recovery tool's default gateway URL. Better: make the recovery tool configurable to use the same routing provider as the API, or provide a recovery-specific resolve endpoint | +| TEE republishing during provider switch | The TEE worker calls the API's `publishSignedRecord()` method, which calls `DelegatedRoutingClient.publish()`. If the URL changes, the TEE's publishes go to the new provider but existing DHT records on the old provider expire | During transition, have the republisher publish to both providers. The TEE itself does not need to change -- only the API's publish path needs to dual-write | +| Prometheus metrics endpoint | Adding IPNS latency histograms that create high-cardinality labels (e.g., per-ipnsName timings) | Use fixed label values (e.g., `operation=publish/resolve`, `source=network/db_cache`). The existing `httpRequestDuration` histogram correctly normalizes routes. Follow the same pattern for new IPNS-specific histograms | + +--- + +## Performance Traps + +Patterns that work at small scale but fail as usage grows. + +| Trap | Symptoms | Prevention | When It Breaks | +| -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Full DHT resolution on every IPNS resolve | Login takes 11+ seconds (DHT median); folder navigation feels frozen | Use DB-cached CID as primary resolve path; DHT resolve as background refresh (the current code approximates this but still blocks on DHT first) | Immediately -- median 11s latency per ProbeLab measurements on Amino DHT. Already broken for UX | +| TEE republishing all records in sequence | Republish cycle exceeds the 6-hour interval for users with many folders/files; records start expiring before the next cycle | The current BATCH_SIZE=100 with concurrency helps. Monitor `republish_entries_processed_total` vs cycle duration. Add per-user prioritization | At ~600+ enrolled records per cycle (100 batches x 6s avg per batch = 600s = 10 min total, still fine; but network issues or TEE latency can push individual batches to 60s+, at which point 100 batches x 60s = 100 min, approaching limits) | +| Instrumenting every IPFS operation with synchronous logging | Console.log/warn calls on hot paths (the codebase has 50+ console calls per CONCERNS.md) block the event loop; adding structured logging to IPFS pin/unpin adds latency to user-facing uploads | Use async log shipping (not synchronous console). Instrument at the histogram level (timing), not the log level (string formatting). The existing Prometheus setup is the right model | Immediately -- any synchronous logging on IPFS hot paths adds measurable latency | +| Client-side performance measurement using `Date.now()` | Sub-millisecond operations appear as 0ms; timer coarsening in browsers hides real variance | Use `performance.now()` for client-side timing, `process.hrtime.bigint()` for server-side (already used in `http-metrics.interceptor.ts:13`). Set histogram buckets appropriately for the expected range | Immediately -- `Date.now()` has millisecond resolution which is insufficient for sub-ms operations | +| Baseline measurements taken during development with hot caches | Baselines show artificially good numbers because Kubo's blockstore cache, OS page cache, and browser cache are warm | Always include cold-start measurements: restart Kubo, clear browser cache, flush OS page cache. Report both cold and warm baselines | When baselines are used to set SLO thresholds -- warm-cache numbers are artificially optimistic | +| BYO-IPFS with user's home node behind NAT | Pin succeeds locally but content is not retrievable by other peers because the node is not reachable. User reports "upload succeeded but file is missing" | Verify content availability after pin by attempting retrieval from a different peer. Warn users about NAT traversal requirements. Kubo's relay/AutoNAT helps but is not guaranteed | Immediately for any user behind consumer NAT (most home connections) | + +--- + +## Security Mistakes + +Domain-specific security issues beyond general web security, relevant to the IPFS infrastructure changes. + +| Mistake | Risk | Prevention | +| ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Exposing Kubo API (port 5001) to BYO users for "direct pinning" | Kubo API has no authentication. Any client with network access can pin/unpin/add content, exhaust disk, or read any pinned content | Never expose Kubo API directly. Use the IPFS Pinning Service API (port-separated, auth-capable) or proxy through the CipherBox API with JWT auth | +| Storing BYO-IPFS node credentials (API tokens, auth headers) in the database without encryption | Server compromise exposes all BYO users' IPFS node credentials | Encrypt BYO config with the user's publicKey (ECIES). Store only the wrapped blob. The client unwraps and uses the credentials client-side | +| Self-hosted Someguy without rate limiting | DoS vector: anyone can flood the delegated routing endpoint with publish/resolve requests, exhausting DHT connections | Deploy Someguy behind a reverse proxy with rate limiting. The existing `delegated-routing.client.ts` handles 429 responses (lines 62-69), so rate-limited responses are gracefully retried | +| Performance baselines that include auth tokens in timing data | Timing side-channels could reveal information about token validation (e.g., shorter response for invalid tokens vs. valid tokens with expired sessions) | Exclude auth endpoints from public-facing performance dashboards. Report only aggregate histograms, not per-request timings. The existing Prometheus setup correctly does this | +| IPNS records signed with leaked TEE epoch keys | If a previous-epoch TEE private key is compromised, an attacker could forge IPNS records for any user whose key was encrypted with that epoch | The 4-week grace period for epoch rotation means old keys are eventually discarded. Ensure the new routing provider validates IPNS record signatures (Someguy does this inherently via the IPNS spec). Monitor for unexpected sequence number regressions | + +--- + +## UX Pitfalls + +Common user experience mistakes when adding these infrastructure features. + +| Pitfall | User Impact | Better Approach | +| ----------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Showing IPNS resolution latency as a loading spinner without explanation | Users see a 3-11 second spinner after every folder navigation and assume the app is broken | Show folder contents from DB cache immediately, then refresh in background when IPNS resolves. The current 30-second polling partially achieves this, but initial navigation still blocks on resolve | +| BYO-IPFS setup requires knowing Kubo API URL, authentication, and IPNS config | Only technically sophisticated users can configure it; everyone else is stuck with default | Provide auto-detection: if user enters a hostname, probe standard ports (5001 for Kubo API, 9097 for Pinning Service API). Offer presets for common providers (web3.storage, Filebase, Pinata) | +| Performance baseline results displayed as raw numbers | Users see "IPNS resolve: 11234ms" and panic, not understanding this is a DHT operation | Present baselines as traffic-light indicators (green/yellow/red) with context: "IPNS resolve: 11.2s (typical for DHT; cached resolves are <100ms)" | +| Silent fallback from IPNS to DB cache with no indication | User thinks they have "real IPFS" but resolution always falls back to the database. False sense of decentralization | Show a subtle indicator when resolution source is DB-cache vs. IPNS network. Log it in a diagnostics panel for advanced users | +| BYO-IPFS node goes offline and user loses access to files | User pinned files to their home node, it crashes, content is lost because no other node has copies | Warn users that BYO means single-point-of-failure unless they also pin to a backup. Offer "pin to both server and BYO" as default mode | + +--- + +## "Looks Done But Isn't" Checklist + +Things that appear complete but are missing critical pieces. + +- [ ] **Routing provider replacement:** New provider resolves IPNS names -- but verify it also PUBLISHES reliably. Resolution can fall back to DB; publishing cannot. Test publish + resolve round-trip, not just resolve. +- [ ] **DB-to-IPFS migration for folder_ipns:** folder_ipns rows exist in IPFS blobs -- but verify the TEE republish service still works. It reads `encrypted_ipns_key` and `sequence_number` from the schedule table, not from IPFS. These columns cannot be eliminated if TEE republishing continues. +- [ ] **BYO-IPFS pin integration:** Files pin to user's node -- but verify they are retrievable by the CipherBox API for serving to other devices. Content must be discoverable on the IPFS network, not just locally pinned. +- [ ] **Performance baselines recorded:** Histograms populated in Prometheus -- but verify baselines include error cases. A P99 that excludes 502/504 errors paints a falsely rosy picture. +- [ ] **Recovery tool updated for new routing:** recovery.html points to new routing endpoint -- but verify it works WITHOUT the CipherBox API running. The entire point of the recovery tool is server-independent vault access. +- [ ] **Orphaned IPNS cleanup:** IPNS records migrated off DB -- but verify the orphaned IPNS records from pre-migration are cleaned up. The CONCERNS.md already flags orphaned IPNS records as tech debt. +- [ ] **Sequence number continuity:** After migration, first publish from client succeeds -- but verify the sequence number is correctly incremented from the pre-migration value, not reset to 1. A reset would cause the DHT to prefer the old (higher-sequence) record. +- [ ] **BYO quota display:** Settings page shows BYO config -- but verify the vault storage usage display updates correctly. If `pinned_cids` is incomplete, the UI shows wrong usage. + +--- + +## Recovery Strategies + +When pitfalls occur despite prevention, how to recover. + +| Pitfall | Recovery Cost | Recovery Steps | +| --------------------------------------------------------------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Sequence number divergence after provider switch | LOW | DB is authoritative. Force-republish all records from DB state to new provider. Client-side sequence numbers in Zustand stores may be stale -- user must refresh (F5). | +| rootFolderKey inaccessible (IPNS resolution failure) | CRITICAL if DB copy removed; LOW if DB copy retained | If DB copy exists: serve from DB (already the fallback). If DB copy was removed: user must use recovery tool with their private key to re-derive IPNS key, resolve from DHT (may be expired), and fetch from IPFS. If both fail: vault is permanently inaccessible. | +| Shares unreachable after migration to IPFS | HIGH | Must revert to DB-backed shares. If DB tables were dropped: reconstruct from IPFS blobs by scanning each user's share IPNS records, but recipient discovery requires the server index to be rebuilt from scratch. | +| BYO concurrency conflict (two devices wrote different metadata) | MEDIUM | Detect by comparing IPNS records from multiple DHT lookups. The record with the higher sequence number wins. The losing device must fetch the winning metadata and re-apply its changes on top (manual merge). No automated merge exists in the current architecture. | +| 48-hour DHT expiry (all records lost from DHT) | MEDIUM | DB-cached CIDs still work for API-mediated resolution. To restore DHT records: trigger emergency republish of all entries. At BATCH_SIZE=100, 1000 records = 10 batches. If each batch takes 30s, full recovery = 5 minutes for TEE signing + publishing time. | +| Quota tracking broken for BYO users | LOW | Run a reconciliation job: for each BYO user, fetch their folder metadata from IPNS, enumerate all CIDs, check pin status on their node, update `pinned_cids` table. Can be run as a one-time migration or periodic background job. | +| False performance baselines from observer effect | LOW | Re-run baselines with instrumentation disabled, compare. If overhead > 5%, switch to sampling-based measurement (10% sample rate). Document the measurement methodology alongside the baseline numbers. | + +--- + +## Pitfall-to-Phase Mapping + +How roadmap phases should address these pitfalls. + +| Pitfall | Prevention Phase | Verification | +| ------------------------------------------ | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| P1: Sequence number divergence | IPNS Reliability | Run dual-provider for 48+ hours; compare sequence numbers across providers for 100% of IPNS names | +| P2: rootFolderKey on IPNS | Database Minimization | Verify login works when IPNS is completely down (kill Someguy, disable DHT); login must still succeed via DB | +| P3: Share discovery on IPFS | Database Minimization | N/A -- prevention is to NOT migrate shares. Verify shares still work after other tables are migrated | +| P4: BYO bypasses concurrency | BYO-IPFS | Simulate two BYO devices publishing simultaneously; verify conflict is detected and one publish is rejected or flagged | +| P5: DHT record expiry during migration | IPNS Reliability | Before cutover, verify new provider has successfully republished all records at least once. Check that zero `stale` entries exist in `ipns_republish_schedule` | +| P6: Quota tracking with BYO | BYO-IPFS | Upload 10 files via BYO, verify `pinned_cids` table has 10 entries with correct sizes. Empty recycle bin, verify CIDs are unpinned on user's node | +| P7: Instrumentation overhead | Performance Baselines | Run baseline suite with and without instrumentation. Verify overhead < 5% on P95 latency. If higher, reduce sampling rate | +| P8: Recovery tool broken by routing change | IPNS Reliability | Run full recovery flow using ONLY recovery.html (no API) with the new routing provider. Verify all files are recoverable | + +--- + +## Sources + +- [IPFS IPNS Concepts](https://docs.ipfs.tech/concepts/ipns/) -- DHT expiry (48h), TTL, republishing behavior +- [Measuring IPNS Performance on the Public Amino DHT](https://www.probelab.network/blog/ipns-performance-amino-dht) -- Median 11s retrieval latency, 100% retrieval success rate +- [Someguy - Delegated Routing V1 server](https://github.com/ipfs/someguy) -- Self-hosted delegated routing, proxies to DHT and IPNI +- [Kubo Delegated Routing docs](https://github.com/ipfs/kubo/blob/master/docs/delegated-routing.md) -- Routing.Type auto/dht/custom configuration +- [IPFS Public Utilities](https://docs.ipfs.tech/concepts/public-utilities/) -- delegated-ipfs.dev as public good endpoint +- [Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- IPIP-476, IPIP-513, Someguy improvements +- [IPFS Pinning Service API spec](https://ipfs.github.io/pinning-services-api-spec/) -- Async pinning, standard API for BYO integration +- [Multiple users publish to IPNS at the same time](https://github.com/ipfs/kubo/issues/8433) -- Concurrent publish conflict, sequence number semantics +- [IPNS Record and Protocol spec](https://specs.ipfs.tech/ipns/ipns-record/) -- Sequence number comparison, record validation +- [Empirical study on performance overhead of code instrumentation (2025)](https://www.sciencedirect.com/science/article/pii/S0164121225002420) -- Up to 8.4% throughput reduction, 20-49% latency increase from instrumentation +- [How to Reduce OpenTelemetry Performance Overhead in Production](https://oneuptime.com/blog/post/2026-02-06-reduce-opentelemetry-performance-overhead-production/view) -- Sampling strategies, batch processing +- [OpenTelemetry NestJS Implementation Guide](https://signoz.io/blog/opentelemetry-nestjs/) -- NestJS-specific OTel setup and sampling +- CipherBox codebase: `apps/api/src/ipns/ipns.service.ts`, `apps/api/src/ipns/delegated-routing.client.ts`, `apps/api/src/republish/republish.service.ts`, `apps/web/src/services/ipns.service.ts`, `apps/web/public/recovery.html` +- CipherBox project context: `.planning/PROJECT.md`, `.planning/codebase/CONCERNS.md`, `.planning/todos/pending/` (IPNS alternatives, BYO-IPFS, move rootFolderKey) + +--- + +_Pitfalls research for: CipherBox v1.1 IPFS Infrastructure_ +_Researched: 2026-03-07_ diff --git a/.planning/research/STACK.md b/.planning/research/_v1.1-archive/STACK.md similarity index 100% rename from .planning/research/STACK.md rename to .planning/research/_v1.1-archive/STACK.md diff --git a/.planning/research/_v1.1-archive/SUMMARY.md b/.planning/research/_v1.1-archive/SUMMARY.md new file mode 100644 index 0000000000..d720bfb65c --- /dev/null +++ b/.planning/research/_v1.1-archive/SUMMARY.md @@ -0,0 +1,187 @@ +# Project Research Summary + +**Project:** CipherBox v1.1 -- IPFS Infrastructure +**Domain:** IPFS/IPNS reliability, database minimization, BYO-IPFS node support, performance baselines +**Researched:** 2026-03-07 +**Confidence:** HIGH + +## Executive Summary + +CipherBox v1.1 is an infrastructure-hardening milestone, not a feature milestone. The four workstreams -- IPNS reliability, database minimization, BYO-IPFS, and performance baselines -- aim to transform CipherBox from "IPFS as a storage backend with database fallbacks" to "IPFS-native with the database serving only coordination and auth." The dominant finding across all four research files is that the existing stack already has the capabilities needed. The Kubo v0.34.0 node can resolve IPNS records locally via its DHT; the `IpfsProvider` interface already abstracts pin/unpin/get for provider extension; `prom-client` is installed with an established `MetricsService` pattern; and TypeORM handles all necessary migrations. **Zero new npm dependencies are required for this entire milestone.** + +The recommended approach is a five-phase execution with strict ordering (Phases 18-22). Phase 18 (performance instrumentation) is zero-risk and establishes server-side baselines before any architectural changes. Phase 19 (IPNS reliability) inverts the resolution model to DB-first with async Kubo DHT verification via self-hosted Someguy, eliminating the `delegated-ipfs.dev` dependency. Phase 20 (database minimization) moves `encryptedRootFolderKey` into an IPFS blob via a versioned vault blob v2 format, with dual-write migration and permanent DB fallback. Phase 21 (BYO-IPFS) adds a `RemotePinningProvider` speaking the standard IPFS Pinning Service API, with dual-pin strategy (always pin to CipherBox node, best-effort mirror to user's node). Phase 22 (performance baselines completion) adds client-side timing instrumentation, end-to-end journey timing, k6 load tests, and capacity documentation after all features are stable. + +The primary risk is in Phase 3: moving rootFolderKey to IPFS creates a hard dependency on IPNS resolution for login. All four research files flag this independently. The mitigation is clear -- keep the DB copy as a permanent fallback, never make IPNS the sole access path for the root folder key. Secondary risks include sequence number divergence during routing provider migration (Phase 2) and quota tracking becoming unenforceable for BYO-IPFS users (Phase 4). Both have well-defined prevention strategies documented in PITFALLS.md. + +## Key Findings + +### Recommended Stack + +The milestone requires zero new npm dependencies. Every capability is built using existing libraries, configuration changes, and new provider implementations. The only external tooling addition is Grafana k6 v1.0 (a standalone Go binary, not an npm package) for load testing scripts. + +**Core technologies (all existing):** + +- **Kubo v0.34.0 (recommend upgrade to v0.40.1):** Self-hosted IPFS node already participating in the Amino DHT. Supports `Gateway.ExposeRoutingAPI` and `Ipns.UsePubsub` for local IPNS resolution. v0.40.1 makes routing API default and improves IPNS-over-PubSub. +- **`prom-client` ^15.1.3:** Already installed with `MetricsService` pattern established. Extend with 4 new histograms for IPFS/IPNS latency tracking. +- **TypeORM ^0.3.28 + PostgreSQL 16.x:** Handles all migration needs. 2-3 new migrations (ADD COLUMN, ALTER COLUMN, CREATE TABLE for user IPFS config). +- **IPFS Pinning Service API (HTTP standard):** Vendor-agnostic OpenAPI spec with 4 endpoints. Implemented by Pinata, Filebase, web3.storage, IPFS Cluster. No SDK needed -- native `fetch` is sufficient. +- **k6 v1.0+ (dev tooling only):** Standalone load test runner with native TypeScript support. Install via `brew install k6`. Not committed to repo. + +### Expected Features + +**Must have (table stakes):** + +- Reliable IPNS resolution (<2s, >99.5% availability) -- current `delegated-ipfs.dev` has documented 502s and stale records +- DB-cached CID fallback with sequence number comparison -- already implemented, needs to become the primary resolution path +- IPNS publish/resolve latency monitoring -- counters exist, need duration histograms +- API endpoint response time baselines with p50/p95/p99 targets +- Graceful degradation when IPFS/IPNS is slow -- timeout + fallback pattern, partially implemented + +**Should have (differentiators):** + +- Move rootFolderKey to IPFS vault record -- server retains DB copy as permanent fallback but primary access shifts to IPFS blob, reducing server's role toward zero-knowledge relay +- BYO-IPFS node support via Pinning Service API -- unique among encrypted storage apps, enables self-sovereignty over data persistence +- End-to-end user journey performance baselines -- login-to-vault, upload-to-visible timing +- IPFS/IPNS latency histograms with per-operation breakdown in Prometheus + +**Defer (v1.2+):** + +- CRDT-based share discovery via IPNS inbox -- research-only in v1.1, architecturally desirable but premature +- Migrate device registry off DB -- requires solving approval handshake without server coordination +- Eliminate `pinned_cids` table -- requires alternative quota tracking +- Client-direct IPFS upload mode -- CORS issues, breaks quota tracking and conflict detection + +### Architecture Approach + +The architecture follows a "DB-first, IPFS-verify" pattern for IPNS resolution, a versioned blob format for metadata evolution, a provider factory pattern for BYO-IPFS extensibility, and additive histogram instrumentation for performance baselines. The key insight is that the DB already serves as the reliable source for IPNS CIDs -- making this explicit (instead of treating it as a fallback) simplifies the architecture and eliminates the external `delegated-ipfs.dev` dependency without adding new infrastructure. + +**Major components:** + +1. **KuboIpnsClient** -- New NestJS injectable wrapping Kubo's RPC API (`/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import`). Handles native DHT resolution and Kubo-native publishing for the TEE republish path. +2. **Vault Blob v2 Format** -- Extends the root IPNS blob with a version byte and ECIES-wrapped `encryptedRootFolderKey` header, enabling client-side key extraction without a DB round-trip. Version-aware reading allows graceful fallback to blob v1. +3. **Provider Factory + DualPinProvider** -- Per-user `IpfsProviderFactory` that returns either the default `LocalProvider` or a `DualPinProvider` wrapping both `LocalProvider` (always) and `UserCustomProvider` (best-effort mirror to user's node). +4. **MetricsService Extensions** -- 4 new Prometheus histograms: IPNS resolve duration, IPNS publish duration, IPFS operation duration, TEE republish batch duration. All use the existing `prom-client` infrastructure. + +### Critical Pitfalls + +1. **Sequence number divergence during routing provider migration** -- The new Kubo DHT endpoint may have no records initially. Run the new provider in read-parallel mode for 48+ hours before cutting writes. Dual-write to old and new providers during transition. Monitor for 409 Conflict error spikes. + +2. **rootFolderKey on IPNS creates login-critical IPNS dependency** -- Moving rootFolderKey exclusively to IPFS means login fails when IPNS is down. The mitigation is absolute: keep the DB copy as the primary source for login, use the IPFS copy for recovery tool independence. This is a "both, not either" situation. Never drop the DB column. + +3. **DHT record expiry during routing provider migration** -- IPNS records expire from the DHT after 48 hours regardless of the validity field. If both providers are misconfigured during a weekend deployment, ALL records expire simultaneously. Build an emergency republish endpoint that bypasses the 6-hour schedule. + +4. **BYO-IPFS bypasses server-side optimistic concurrency** -- If users publish IPNS records directly to their node (bypassing the API), the server cannot check sequence numbers. The fix: BYO-IPFS affects ONLY where data is pinned, never how metadata is published. All IPNS publishes must still go through the CipherBox API. + +5. **Quota tracking becomes unenforceable with BYO-IPFS** -- Server cannot meter uploads it does not relay. For BYO users, skip server-side quota enforcement and require client-reported CID sizes. Mark `pinned_cids` entries with a `provider` column to distinguish server-pinned from BYO-pinned. + +## Implications for Roadmap + +Based on research, suggested phase structure: + +### Phase 1: Performance Instrumentation + +**Rationale:** Zero risk to existing functionality. Purely additive. Establishes "before" measurements that Phase 2-4 can be compared against. Without baselines, we cannot prove that subsequent phases improve performance or detect regressions. +**Delivers:** 4 new Prometheus histograms (IPNS resolve/publish duration, IPFS operation duration, TEE republish batch duration), client-side timing utility, baseline measurements document, k6 load test scripts. +**Addresses:** Table stakes features -- IPNS publish latency monitoring, API endpoint baselines. +**Avoids:** Pitfall P7 (instrumentation overhead) -- verify overhead < 5% on P95 latency with and without instrumentation. +**Estimated scope:** Small. ~10 modified files, no new entities or migrations. + +### Phase 2: IPNS Resolution Improvement + +**Rationale:** Fixes the most critical reliability issue and is the gating dependency for Phase 3. The current `delegated-ipfs.dev` dependency causes 502 errors and stale records. Must be resolved before making IPNS a login-critical path. +**Delivers:** `KuboIpnsClient` for native Kubo DHT resolution, DB-first resolution strategy, `delegated-ipfs.dev` demoted to disabled-by-default fallback, Kubo config with `Ipns.UsePubsub: true`, updated recovery tool resolution chain. +**Addresses:** Table stakes -- reliable IPNS resolution, graceful degradation. +**Avoids:** Pitfall P1 (sequence number divergence) -- run dual-provider for 48+ hours, compare sequence numbers across providers. Pitfall P5 (DHT record expiry) -- never take old provider offline until new one has processed a full republish cycle. +**Estimated scope:** Medium. 1 new file, ~5 modified files, Kubo config change, recovery tool update. + +### Phase 3: Database Minimization (rootFolderKey to IPFS) + +**Rationale:** Requires reliable IPNS resolution from Phase 2. This is the highest-value zero-knowledge improvement: the server stores zero crypto material after migration. The vault blob v2 format is a breaking metadata change requiring careful migration across web, desktop, and recovery tool. +**Delivers:** Vault blob v2 format with embedded `encryptedRootFolderKey`, version-aware blob reading, dual-write migration strategy, `encryptedRootIpnsPrivateKey` deprecation, updated recovery tool, metadata schema version bump per METADATA_EVOLUTION_PROTOCOL. +**Addresses:** Differentiator -- server stores zero crypto material, true zero-knowledge relay. +**Avoids:** Pitfall P2 (rootFolderKey on IPNS) -- keep DB copy as permanent fallback for login, use IPFS copy for recovery tool independence. Pitfall P3 (share discovery on IPFS) -- explicitly do NOT migrate shares, keep sharing graph in DB. +**Estimated scope:** Large. New type definitions, serialization code, changes across 3 clients (web, desktop, recovery). + +### Phase 4: BYO-IPFS Node Support + +**Rationale:** Largest surface area but most independent feature. Benefits from stable IPNS resolution (Phase 2) and reduced DB dependency (Phase 3) but does not strictly require them. The `RemotePinningProvider` is additive -- it mirrors pins without replacing the local provider. +**Delivers:** `UserCustomProvider` (Kubo RPC + Pinning Service API), `DualPinProvider` wrapper, `ProviderFactory` for per-user resolution, `user_ipfs_config` entity and migration, IPFS config CRUD API endpoints, connection test endpoint, Settings page UI for IPFS node configuration, BYO-specific Prometheus metrics. +**Addresses:** Differentiator -- BYO-IPFS node support, unique among encrypted storage apps. +**Avoids:** Pitfall P4 (BYO bypasses concurrency) -- all IPNS publishes still go through API, BYO affects only pinning. Pitfall P6 (quota tracking) -- skip server-side quota for BYO users, require client-reported CID sizes, add `provider` column to `pinned_cids`. +**Estimated scope:** Large. New entity, migration, 4+ new provider files, UI component, multiple API endpoints. + +### Phase Ordering Rationale + +- **Phase 1 before everything:** Baselines must exist before changes, or you cannot measure impact. Zero dependencies, zero risk. +- **Phase 2 before Phase 3 (hard dependency):** Moving rootFolderKey to IPFS makes IPNS resolution login-critical. IPNS must be reliable first. All four research files independently flag this as the most important ordering constraint. +- **Phase 3 before Phase 4 (soft dependency):** Not strictly required, but Phase 3 reduces DB dependency which aligns with BYO-IPFS goals. Phase 4's design work can proceed in parallel with Phase 3's implementation. +- **Phase 4 last:** Largest surface area, most new code, benefits from all prior infrastructure improvements. + +### Research Flags + +Phases likely needing deeper research during planning: + +- **Phase 3 (Database Minimization):** Migration protocol edge cases -- blob v2 format across 3 clients, dual-write/dual-read window management, recovery tool independence, desktop (Rust) blob parsing. This is the riskiest phase. +- **Phase 4 (BYO-IPFS):** Per-user provider routing in NestJS DI, IPFS Pinning Service API real-world testing, auth token storage model (ECIES-wrapped vs server-encrypted), Settings UI design. + +Phases with standard patterns (skip research-phase): + +- **Phase 1 (Performance Instrumentation):** Established `prom-client` + `MetricsService` pattern. k6 is well-documented. Purely additive. +- **Phase 2 (IPNS Resolution):** Kubo RPC API is documented, DB-first resolution is architecturally straightforward. The main risk (sequence number divergence during transition) is an operational concern, not a research gap. + +## Confidence Assessment + +| Area | Confidence | Notes | +| ------------ | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Stack | HIGH | Zero new npm dependencies. All capabilities use existing libraries. Kubo API contracts verified against official docs. | +| Features | HIGH for IPNS/perf, MEDIUM for DB migration and BYO-IPFS | IPNS reliability and performance monitoring are well-documented domains. DB-to-IPFS migration has app-specific edge cases. BYO-IPFS Pinning Service API is well-specified but per-user provider routing patterns are sparse. | +| Architecture | HIGH | Existing codebase analyzed in detail. DB-first resolution, provider factory, blob versioning are all established patterns. Build order validated against dependency graph. | +| Pitfalls | HIGH | 6 critical pitfalls identified with specific prevention strategies, warning signs, and recovery plans. Phase-to-pitfall mapping is complete. | + +**Overall confidence:** HIGH + +### Gaps to Address + +- **rootFolderKey migration dual-write window:** How long should the dual-write period last? What happens to users who never log in during the window? Need a forced migration strategy (background job that reads DB key, writes blob v2, publishes to IPNS) for dormant accounts. +- **BYO-IPFS auth token storage model:** STACK.md recommends server-side encryption (server needs to decrypt to call user's node). ARCHITECTURE.md implements this. PITFALLS.md warns about server compromise exposing tokens. The tradeoff (server sees token but not plaintext content) needs explicit acceptance and documentation. +- **Kubo version decision:** v0.34.0 works but v0.40.1 is recommended. The upgrade should happen before Phase 2 starts but is not blocking. Need to validate that Kubo v0.40.1 does not introduce breaking changes for the existing `LocalProvider` calls. +- **CRDT inbox for shares:** Research-only this milestone. Need to document the CRDT approach as a design RFC during Phase 3 work so it feeds into v1.2 planning. +- **Recovery tool independence:** The recovery tool currently resolves IPNS via `delegated-ipfs.dev` directly from the browser. Phases 2 and 3 both modify its behavior. Need to verify it works WITHOUT the CipherBox API running after all changes. +- **IPNS routing approach reconciliation:** STACK.md recommends Kubo's `Gateway.ExposeRoutingAPI`, FEATURES.md recommends self-hosted Someguy, ARCHITECTURE.md recommends DB-first with Kubo RPC `/api/v0/name/resolve`. Adopt the ARCHITECTURE.md approach (DB-first with async Kubo DHT verification) as the definitive strategy -- it is the most robust because it makes the already-reliable DB the primary path and uses Kubo DHT only for background verification. + +## Sources + +### Primary (HIGH confidence) + +- [Delegated Routing V1 HTTP API Spec](https://specs.ipfs.tech/routing/http-routing-v1/) -- IPNS endpoint contract +- [IPFS Pinning Service API Spec](https://ipfs.github.io/pinning-services-api-spec/) -- BYO-IPFS integration standard +- [Kubo RPC API v0 Reference](https://docs.ipfs.tech/reference/kubo/rpc/) -- `/api/v0/name/resolve`, `/api/v0/name/publish`, `/api/v0/key/import` +- [Kubo Configuration Reference](https://github.com/ipfs/kubo/blob/master/docs/config.md) -- `Ipns.UsePubsub`, `Routing.Type`, `Gateway.ExposeRoutingAPI` +- [Kubo v0.34.0](https://github.com/ipfs/kubo/releases/tag/v0.34.0) and [v0.40.0](https://github.com/ipfs/kubo/releases/tag/v0.40.0) Release Notes +- [Someguy GitHub](https://github.com/ipfs/someguy) -- Self-hosted delegated routing reference +- [Grafana k6 1.0 Release](https://grafana.com/blog/grafana-k6-1-0-release/) -- Load testing +- [prom-client GitHub](https://github.com/siimon/prom-client) -- Prometheus client for Node.js +- [IPNS Record and Protocol spec](https://specs.ipfs.tech/ipns/ipns-record/) -- Sequence number semantics + +### Secondary (MEDIUM confidence) + +- [ProbeLab IPFS KPIs](https://www.probelab.io/ipfs/kpi/) and [Week 07, 2026 Results](https://discuss.ipfs.tech/t/probelabs-notable-ipfs-performance-results-week-07-2026/20048) -- DHT performance baselines +- [IPIP-0379: Delegated IPNS HTTP API](https://specs.ipfs.tech/ipips/ipip-0379/) -- IPNS delegation spec +- [IP Shipyard 2025 Year in Review](https://ipshipyard.com/blog/2025-shipyard-ipfs-year-in-review/) -- IPFS ecosystem context +- [Measuring IPNS Performance on the Public Amino DHT](https://www.probelab.network/blog/ipns-performance-amino-dht) -- Median 11s retrieval latency reference +- [Empirical study on performance overhead of code instrumentation (2025)](https://www.sciencedirect.com/science/article/pii/S0164121225002420) -- Instrumentation overhead benchmarks + +### Codebase (HIGH confidence) + +- `apps/api/src/ipns/ipns.service.ts` -- Current IPNS resolution logic with DB fallback +- `apps/api/src/ipns/delegated-routing.client.ts` -- Current routing client (3 retries, 10s timeout) +- `apps/api/src/ipfs/providers/ipfs-provider.interface.ts` -- 3-method provider abstraction +- `apps/api/src/metrics/metrics.service.ts` -- Existing Prometheus infrastructure +- `apps/api/src/republish/republish.service.ts` -- TEE republish via delegated routing +- `apps/web/public/recovery.html` -- Standalone recovery tool with direct IPNS resolution +- `docs/METADATA_SCHEMAS.md` and `docs/METADATA_EVOLUTION_PROTOCOL.md` -- Schema migration rules + +--- + +_Research completed: 2026-03-07_ +_Ready for roadmap: yes_ From 19fced95db2e2c0910c32d2374011fb81dcdc0c9 Mon Sep 17 00:00:00 2001 From: Michael Yankelev Date: Sat, 27 Jun 2026 02:40:51 +0200 Subject: [PATCH 5/5] docs: tag 1 pending todo with resolves_phase after milestone v2.0 roadmap Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01SDNQVLoAw4DbPrPvtQPobh Entire-Checkpoint: aecd4e95cb28 --- ...bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.planning/todos/pending/2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md b/.planning/todos/pending/2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md index 689df1612f..4e9fe7cb07 100644 --- a/.planning/todos/pending/2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md +++ b/.planning/todos/pending/2026-06-23-bin-delete-and-empty-bin-leak-content-and-version-cid-pins.md @@ -12,6 +12,7 @@ files: - apps/api/src/shares/shares.service.ts - apps/api/src/shares/shares.controller.ts - apps/api/src/shares/dto/revoke-for-items.dto.ts +resolves_phase: 65 --- ## Problem