diff --git a/.claude/claude.md b/.claude/claude.md index db9fc49d3f..3994a25163 100644 --- a/.claude/claude.md +++ b/.claude/claude.md @@ -6,14 +6,13 @@ CipherBox is a **technology demonstrator** for privacy-first encrypted cloud sto ## Documentation Structure -| Document | Purpose | -| ------------------------------------------------------------ | ------------------------------------------ | -| `00-Preliminary-R&D/Documentation/PRD.md` | Product requirements, user journeys, scope | -| `00-Preliminary-R&D/Documentation/TECHNICAL_ARCHITECTURE.md` | Encryption, key hierarchy, system design | -| `00-Preliminary-R&D/Documentation/API_SPECIFICATION.md` | Backend endpoints, database schema | -| `00-Preliminary-R&D/Documentation/DATA_FLOWS.md` | Sequence diagrams, test vectors | -| `00-Preliminary-R&D/Documentation/CLIENT_SPECIFICATION.md` | Web UI, desktop app specs | -| `00-Preliminary-R&D/Documentation/IMPLEMENTATION_ROADMAP.md` | Week-by-week development plan | +| Document | Purpose | +| ----------------------------------------------------------------- | ------------------------------------------ | +| `.planning/milestones/m0/Documentation/PRD.md` | Product requirements, user journeys, scope | +| `.planning/milestones/m0/Documentation/TECHNICAL_ARCHITECTURE.md` | Encryption, key hierarchy, system design | +| `.planning/milestones/m0/Documentation/API_SPECIFICATION.md` | Backend endpoints, database schema | +| `.planning/milestones/m0/Documentation/DATA_FLOWS.md` | Sequence diagrams, test vectors | +| `.planning/milestones/m0/Documentation/CLIENT_SPECIFICATION.md` | Web UI, desktop app specs | ## Finalized Specifications @@ -22,12 +21,12 @@ CipherBox is a **technology demonstrator** for privacy-first encrypted cloud sto ### ⚠️ IMPORTANT: Do Not Edit Preliminary/Documentation Files -All files in `00-Preliminary-R&D/Documentation/` are **FINALIZED** specifications (version 1.10.0, status: Finalized). These documents represent the agreed-upon design and should **NOT** be modified. +All files in `.planning/milestones/m0/Documentation/` are **FINALIZED** specifications (version 1.10.0, status: Finalized). These documents represent the agreed-upon design and should **NOT** be modified. **If you need to make changes:** - New implementation documentation should be created in a separate location - Working notes and updates belong in `.planning/` or project-specific directories -- Do not modify version numbers or content in `00-Preliminary-R&D/Documentation/` unless explicityly authorized. +- Do not modify version numbers or content in `.planning/milestones/m0/Documentation/` unless explicitly authorized. ## Terminology Standards @@ -91,7 +90,7 @@ When generating code for CipherBox: - **Encryption:** Client-side only, server is zero-knowledge - **Sync:** IPNS polling (30s interval), no push infrastructure - **Desktop:** FUSE mount for transparent file access -- **TEE Republishing:** Phala Cloud (primary) / AWS Nitro (fallback) for automatic IPNS republishing every 3 hours +- **TEE Republishing:** Phala Cloud CVM for automatic IPNS republishing every 6 hours - **Key Epochs:** TEE public keys rotate with 4-week grace period for seamless migration ## Out of Scope (v1.0) diff --git a/.planning/DEFERRED.md b/.planning/DEFERRED.md index cdec91dbe7..8bd8aa8f51 100644 --- a/.planning/DEFERRED.md +++ b/.planning/DEFERRED.md @@ -1,22 +1,21 @@ # Deferred Items Inventory -**Last updated:** 2026-03-29 +**Last updated:** 2026-03-31 -Items deferred across milestones v1.0 (phases 11-17.1) and v1.1 (phases 18-27). +Items deferred across milestones v1.0 (phases 11-17.1) and v1.1 (phases 18-37). Cross-referenced with `.planning/todos/pending/` and security review findings. ## Active Pending Todos These are explicitly tracked in `.planning/todos/pending/`: -| Date | Item | Priority | -| ---------- | ----------------------------------------------------------------------------- | -------- | -| 2026-02-07 | Offload large file encryption to Web Worker (files >= 10MB block main thread) | Medium | -| 2026-02-14 | ERC-1271 contract wallet authentication (Safe, Argent, Sequence) | Low | -| 2026-02-22 | CRDT-based IPNS inbox for serverless share discovery | Research | -| 2026-02-24 | Make search index build async/incremental for large vaults | Medium | -| 2026-02-26 | Alternative MFA factor types (passkeys, password-derived) | Medium | -| 2026-03-23 | Investigate removal of mock-ipns-routing layer (someguy works now) | Low | +| Date | Item | Priority | +| ---------- | ---------------------------------------------------------------- | -------- | +| 2026-02-14 | ERC-1271 contract wallet authentication (Safe, Argent, Sequence) | Low | +| 2026-02-22 | CRDT-based IPNS inbox for serverless share discovery | Research | +| 2026-02-24 | Make search index build async/incremental for large vaults | Medium | +| 2026-02-26 | Alternative MFA factor types (passkeys, password-derived) | Medium | +| 2026-03-23 | Investigate removal of mock-ipns-routing layer (someguy works) | Low | ## Security Review Findings (Deferred from Phase 14) @@ -39,7 +38,7 @@ From `.planning/todos/done/2026-02-21-phase14-security-review-deferred.md`: | Attribution / audit trail (`lastModifiedBy` in metadata) | 27 | Track who modified what in shared folders | | Transitive re-sharing | 27 | Allow recipients to share onward; needs cascading revocation | | Share notifications (permission changes) | 14, 27 | Notify recipients of upgrade/downgrade/revoke | -| User discovery service (by email/username/wallet) | 14 | Privacy controls needed; separate phase | +| User discovery service (by email/username/wallet) | 14 | Public key lookup exists; email/username discovery not built | | Display names for share recipients | 14 | Depends on user discovery/profile | | Immediate key rotation on revoke | 14, 27 | Currently lazy; more secure but requires re-wrapping | | CRDT-based IPNS inbox | 14 | Decentralized share discovery replacing `shares` table | @@ -47,18 +46,16 @@ From `.planning/todos/done/2026-02-21-phase14-security-review-deferred.md`: ### Desktop Platform -| Item | Source Phase | Notes | -| ---------------------------------------------------------------- | ------------ | ------------------------------------------------------------- | -| Desktop sharing UI | 14 | No share dialog in desktop app | -| Desktop recycle bin UI | 17 | Bin operations web-only; desktop has no bin browsing | -| Desktop search | 15.1 | No search in desktop app | -| Desktop device approval polling | 11.1 | `approveDevice()` API-complete but no desktop notification UI | -| Desktop FUSE CTR streaming support | 12.1 | Web has CTR playback; desktop update deferred | -| Desktop .Trash folder integration | 17 | Finder/Explorer native trash integration | -| Platform code signing (Apple notarization, Windows Authenticode) | 25 | Required for production distribution | -| Beta/canary update channels | 25 | Future if needed | -| Delta updates | 25 | Tauri supports but adds complexity | -| Linux FUSE mount | 11.3 | Implemented but less tested than macOS | +| Item | Source Phase | Notes | +| ------------------------------------------ | ------------ | --------------------------------------------------------------------------- | +| Desktop sharing UI | 14 | No share dialog in desktop app (FUSE-only, no file browser) | +| Desktop recycle bin UI | 17 | Bin operations web-only; desktop has no bin browsing | +| Desktop search | 15.1 | No search in desktop app | +| Desktop device approval polling | 11.1 | Core polling logic exists; post-auth always-on listener not yet implemented | +| Desktop .Trash folder integration | 17 | Finder/Explorer native trash integration | +| Platform code signing (Apple notarization) | 25 | Windows signing configured; macOS notarization not yet set up | +| Beta/canary update channels | 25 | Single release channel only | +| Delta updates | 25 | Tauri supports but adds complexity | ### Authentication & Security @@ -71,23 +68,41 @@ From `.planning/todos/done/2026-02-21-phase14-security-review-deferred.md`: ### Performance & Infrastructure -| Item | Source Phase | Notes | -| ------------------------------------ | ------------ | -------------------------------------------------- | -| Web Worker for large file encryption | - | Files >= 10MB block main thread | -| Async/incremental search index | 15.1 | `buildFromFolderTree()` blocks UI for large vaults | -| BYO IPFS provider benchmarks | 21 | Requires external provider infrastructure | -| Automated CI timing gates | 26 | Flaky due to runner variance | -| Remove mock-ipns-routing | 19 | Someguy at :8190 may replace it | -| Push notifications (WebSocket sync) | 16 | Currently polling-only; requires backend infra | +| Item | Source Phase | Notes | +| ---------------------------------------------- | ------------ | ------------------------------------------------------- | +| Async/incremental search index | 15.1 | `buildFromFolderTree()` blocks UI for large vaults | +| BYO IPFS provider benchmarks | 21 | Requires external provider infrastructure | +| Automated CI timing gates | 26 | Flaky due to runner variance | +| Remove mock-ipns-routing | 19 | Someguy at `:8190` may replace it | +| Push notifications (WebSocket sync) | 16 | Currently polling-only; requires backend infra | +| Batch API endpoint for IPNS resolves | 32 | Could reduce round trips for folders with many files | +| Kubo API access control (reverse proxy or ACL) | 29 | Current Docker 127.0.0.1 binding sufficient for staging | + +### Upload Pipeline (Phase 37) + +| Item | Source Phase | Notes | +| ----------------------------------------------- | ------------ | ---------------------------------------------------------------------------------- | +| Adaptive concurrency based on file size | 37 | Fixed pool of 3 is sufficient; adaptive sizing adds complexity | +| FUSE write-coalescing for desktop batch uploads | 37 | Desktop uploads arrive one-at-a-time via `release()`; FUSE has no batch context | +| Accumulated retry batching | 37 | Batch retries into single folder publish instead of N individual publishes | +| AbortSignal support for in-flight batch uploads | 37 | No way to cancel once `uploadFiles()` invoked; needs AbortSignal through p-limit | +| Lazy file reading within concurrency pool | 37 | `useDropUpload` reads all files upfront; SDK needs `File` objects or read callback | + +### Observability (Phases 28, 30) + +| Item | Source Phase | Notes | +| --------------------------------------- | ------------ | ---------------------------------------- | +| `no-console` ESLint rule enforcement | 28 | Optional enforcement mechanism | +| Web Worker logging (MessagePort bridge) | 28 | Requires separate communication protocol | +| "Report a problem" user-facing button | 30 | Nice-to-have, not in scope | ### Sync & Conflict Resolution (Deferred to Milestone 4) -| Item | Source Phase | Notes | -| -------------------------------------------- | ------------ | ------------------------------------------ | -| Offline operation queue (IndexedDB) | 16 | Persist writes for replay on reconnect | -| Idempotent replay | 16 | Idempotency keys for queued operations | -| Auto-merge of non-conflicting folder changes | 16 | Three-way merge on encrypted metadata | -| Per-file IPNS conflict detection | 16 | Currently covered by versioning safety net | +| Item | Source Phase | Notes | +| -------------------------------------------- | ------------ | -------------------------------------- | +| Offline operation queue (IndexedDB) | 16 | Persist writes for replay on reconnect | +| Idempotent replay | 16 | Idempotency keys for queued operations | +| Auto-merge of non-conflicting folder changes | 16 | Three-way merge on encrypted metadata | ### Data Management @@ -95,35 +110,43 @@ From `.planning/todos/done/2026-02-21-phase14-security-review-deferred.md`: | --------------------------------------------- | ------------ | ------------------------------------------------------------------ | | TEE unenrollment on file/folder delete | 12.6, 17 | Orphaned IPNS records expire naturally (24h) but waste TEE compute | | TEE enrollment drift reconciliation | 12.6 | Periodic vault scan to sync enrollment | -| Column DROP migration (vault v1 fields) | 20 | After all users migrated, drop legacy columns | -| User-configurable bin retention period | 17 | Currently fixed 30-day retention | +| User-configurable bin retention period | 17 | End-user setting; operator env var exists but no per-user control | | Retroactive TEE enrollment for existing files | 25 | New files only; existing files not enrolled | +| Periodic reconciliation job for unenrollment | 29 | Fire-and-forget pattern may be insufficient | ### Code Quality -| Item | Source Phase | Notes | -| ----------------------------------------------- | ------------ | ------------------------------------- | -| DTS circular build dependency (crypto <-> core) | 19.1 | Workaround in place; not fixed | -| Desktop FUSE automated tests | - | Manual testing only (see CONCERNS.md) | +| Item | Source Phase | Notes | +| -------------------------------------------- | ------------ | ---------------------------------------------------------------------- | +| Full retirement of folder.service.ts | 31 | 1,059 lines, 9 importers; migrate callers to SDK methods | +| Full retirement of bin.service.ts | 31 | 971 lines, only `initializeBin` + `purgeExpired` still used by 2 hooks | +| Remove crypto -> core circular devDependency | 19.1 | Test-only import; refactor vault-ipns test to use hardcoded vectors | ## Items Implemented in Later Phases These were deferred but have since been completed: -| Item | Deferred From | Implemented In | -| -------------------------------------- | ------------------------ | -------------- | -| File versioning | v1.0 scope exclusion | Phase 13 | -| User-to-user sharing | v1.0 scope exclusion | Phase 14 | -| Read-write sharing | Phase 14 | Phase 27 | -| Per-file IPNS metadata | Phase 12 | Phase 12.6 | -| SDK extraction | Phase 11 | Phase 19.1 | -| Rust SDK extraction | Phase 19.1 | Phase 23 | -| BYO IPFS node support | Phase 12.1 | Phase 21 | -| Vault key blob (zero-knowledge server) | Phase 12 | Phase 20 | -| Client-side search | Phase 15 | Phase 15.1 | -| Performance baselines | Phase 18 | Phase 22 | -| Link sharing | Phase 14 | Phase 15 | -| Pagination on shares endpoints (L4) | Phase 14 security review | Phase 14 | -| Structured logging wrapper for web app | - | Phase 28 | - - +| Item | Deferred From | Implemented In | +| ----------------------------------------- | ------------------------ | -------------- | +| File versioning | v1.0 scope exclusion | Phase 13 | +| User-to-user sharing | v1.0 scope exclusion | Phase 14 | +| Read-write sharing | Phase 14 | Phase 27 | +| Per-file IPNS metadata | Phase 12 | Phase 12.6 | +| SDK extraction | Phase 11 | Phase 19.1 | +| Rust SDK extraction | Phase 19.1 | Phase 23 | +| BYO IPFS node support | Phase 12.1 | Phase 21 | +| Vault key blob (zero-knowledge server) | Phase 12 | Phase 20 | +| Client-side search | Phase 15 | Phase 15.1 | +| Performance baselines | Phase 18 | Phase 22 | +| Link sharing | Phase 14 | Phase 15 | +| Pagination on shares endpoints (L4) | Phase 14 security review | Phase 14 | +| Structured logging wrapper for web app | - | Phase 28 | +| Web Worker for large file encryption | - | Phase 37 | +| Error tracking (Grafana Faro) | Phase 28, 30 | Phase 30 | +| Desktop FUSE CTR streaming | Phase 12.1 | Phase 12.1 | +| Linux FUSE mount | Phase 11.3 | Phase 11.3 | +| Per-file IPNS conflict detection | Phase 16 | Phase 12.6 | +| Batch upload secondary pin warning events | Phase 37 | Phase 37 | +| Remote log shipping (Grafana Faro) | Phase 28 | Phase 30 | + + diff --git a/.planning/FEATURES.md b/.planning/FEATURES.md index 48d44977f0..16684b61e9 100644 --- a/.planning/FEATURES.md +++ b/.planning/FEATURES.md @@ -1,6 +1,6 @@ # CipherBox Feature Matrix -**Last updated:** 2026-03-28 +**Last updated:** 2026-03-30 ## Platform Feature Matrix @@ -22,9 +22,10 @@ | Link/unlink auth methods | Y | - | Y | - | - | | **File Operations** | | | | | | | Upload (single file) | Y | Y | Y | Y | full-workflow | +| Upload (batch pipeline) | Y | - | Y | Y | batch-upload (SDK) | | Upload (drag-and-drop) | Y | - | - | - | full-workflow | | Download (single) | Y | Y | Y | Y | full-workflow | -| Download (batch zip) | Y | - | - | - | - | +| Download (batch selection) | Y | - | - | - | batch-download | | Rename | Y | Y | - | Y | full-workflow | | Delete (to bin) | Y | Y | - | Y | recycle-bin | | Move (between folders) | Y | Y | - | Y | full-workflow | @@ -69,10 +70,10 @@ | Cmd/Ctrl+K shortcut | Y | - | - | - | search-workflow | | Keyboard navigation | Y | - | - | - | search-workflow | | **Media Preview** | | | | | | -| Image preview | Y | - | - | - | full-workflow | -| PDF viewer | Y | - | - | - | - | -| Video player (streaming) | Y | - | - | - | - | -| Audio player | Y | - | - | - | - | +| Image preview | Y | - | - | - | media-preview | +| PDF viewer | Y | - | - | - | media-preview | +| Video player (streaming CTR) | Y | - | - | - | streaming-playback | +| Audio player | Y | - | - | - | media-preview | | **Sync** | | | | | | | IPNS polling (30s) | Y | Y | - | Y | conflict-detection | | Conflict detection (409) | Y | Y | Y | Y | conflict-detection | @@ -84,6 +85,14 @@ | OS keychain storage | - | Y | - | - | - | | Auto-updater | - | Y | - | - | - | | Dev-key mode (headless) | - | Y | - | - | desktop-e2e | +| **Vault Settings (Phase 39)** | | | | | | +| Bin retention period | - | - | - | - | - | +| Delete behavior (soft/hard) | - | - | - | - | - | +| Max versions per file | - | - | - | - | - | +| Version cooldown period | - | - | - | - | - | +| **Encryption** | | | | | | +| Web Worker encryption | Y | - | - | - | - | +| AES-CTR streaming decryption | Y | - | - | Y | streaming-playback | | **Infrastructure** | | | | | | | TEE IPNS republishing | - | - | Y | - | - | | BYO IPFS node support | - | - | Y | - | sdk-e2e | @@ -91,12 +100,15 @@ | Prometheus metrics | - | - | Y | - | - | | Vault recovery tool | Y | - | - | - | recovery | | Performance baselines | - | - | Y | - | journey-timing | +| Structured logging (web) | Y | - | - | - | - | ## E2E Test Suites +### Web E2E (`tests/web-e2e/`) + | Suite | File | Coverage | | ------------------ | ------------------------------ | --------------------------------------------------------------------------------------------------- | -| Full Workflow | `full-workflow.spec.ts` | Login, vault init, folders, files, edit, rename, move, delete, versioning, media preview | +| Full Workflow | `full-workflow.spec.ts` | Login, vault init, folders, files, edit, rename, move, delete, versioning | | Sharing | `sharing-workflow.spec.ts` | Direct shares, multi-recipient, revocation, key rotation, hide | | Writable Shares | `writable-shares.spec.ts` | Write permission, recipient uploads/mkdir/rename/delete, permission upgrade/downgrade, file editing | | Invite Links | `invite-link-workflow.spec.ts` | Create invite, claim, revoke, landing page | @@ -107,17 +119,44 @@ | Recovery | `recovery.spec.ts` | Vault recovery tool via IPFS-direct v2 blob path | | Conflict Detection | `conflict-detection.spec.ts` | Multi-device conflicts, auto-resync | | Journey Timing | `journey-timing.spec.ts` | Performance benchmarks for critical paths | -| Load Test | `load-test.spec.ts` | Concurrent operations stress testing | +| Batch Download | `batch-download.spec.ts` | Multi-file selection, sequential individual downloads | +| Media Preview | `media-preview.spec.ts` | Image, PDF, video, audio preview dialogs | +| Streaming Playback | `streaming-playback.spec.ts` | AES-CTR streaming for large videos, GCM fallback for small videos | + +### SDK E2E (`tests/sdk-e2e/`) + +| Suite | File | Coverage | +| --------------------- | ------------------------------- | --------------------------------------------- | +| Vault Lifecycle | `vault-lifecycle.test.ts` | Init, key derivation, destroy | +| Folder CRUD | `folder-crud.test.ts` | Create, rename, move, delete folders | +| File Operations | `file-operations.test.ts` | Upload, download, rename, delete files | +| Batch Upload | `batch-upload.test.ts` | `uploadFiles()` multi-file pipeline | +| Bin Operations | `bin-operations.test.ts` | Soft delete, restore, permanent delete, empty | +| Share Operations | `share-operations.test.ts` | Direct shares, revocation, key re-wrapping | +| Invite Link | `invite-link.test.ts` | Create, claim, revoke invite links | +| Concurrent Operations | `concurrent-operations.test.ts` | Parallel uploads, downloads, folder ops | +| Data Integrity | `data-integrity.test.ts` | Round-trip encryption/decryption verification | +| Error Cases | `error-cases.test.ts` | Invalid inputs, network failures, edge cases | +| IPNS Consistency | `ipns-consistency.test.ts` | Sequence numbers, conflict detection | + +### Load Tests (`tests/load/`) + +| Type | Coverage | +| -------------- | ------------------------------------------ | +| Spike Test | Sudden burst of concurrent operations | +| Throughput | Sustained upload/download rate measurement | +| Mixed Workload | Combined CRUD operations under load | +| Sustained Load | Extended duration stability testing | +| BYO Ceiling | BYO-IPFS provider capacity limits | ## Features Without E2E Coverage -- PDF viewer, video player, audio player (manual testing only) -- Batch download (zip) - Link/unlink auth methods - Device registry sync - OS keychain storage - Auto-updater - TEE republishing - Pin migration +- Web Worker encryption (unit tested, not E2E) - + diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 0ffc64cdb0..fd35f05dbf 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -466,6 +466,10 @@ Phases execute in numeric order: 18 -> 19 -> 19.1 -> 19.2 -> 20 -> 21 -> 22 -> 2 | 33. Windows Async FilePointer Resolution | v1.1 | 2/2 | Complete | 2026-03-28 | | 34. E2E Test Expansion & Baselines | v1.1 | 4/4 | Complete | 2026-03-29 | | 35. Phala Testnet TEE Migration | v1.1 | 6/6 | Complete | 2026-03-29 | +| 36. Inline Upload Progress | post-v1.1 | 2/2 | Complete | 2026-03-29 | +| 37. Parallel Batch Upload Pipeline | post-v1.1 | 2/2 | Complete | 2026-03-30 | +| 38. Retire Deprecated Web Services | post-v1.1 | - | Planned | - | +| 39. User-Configurable Vault Parameters | post-v1.1 | - | Planned | - | ### Phase 27: Writable Shares (PoC) @@ -513,8 +517,24 @@ Plans: - [x] 37-01-PLAN.md -- SDK batch uploadFiles() method with p-limit concurrency pool, stale-children re-read, partial failure handling - [x] 37-02-PLAN.md -- Web Worker encryption offloading, EncryptionWorkerService wrapper, useDropUpload rewire to batch API +### Phase 38: Retire deprecated web services + +**Goal:** Remove `folder.service.ts` (1,059 lines) and `bin.service.ts` (971 lines) by migrating all remaining callers to `@cipherbox/sdk` methods, eliminating the deprecated service layer. Also remove the circular devDependency from `@cipherbox/crypto` on `@cipherbox/core` by refactoring the vault-ipns test to use hardcoded test vectors instead of cross-package imports. +**Requirements**: None (tech debt cleanup, deferred from Phase 31) +**Depends on:** Phase 37 + +Plans: _not yet planned_ + +### Phase 39: User-configurable vault parameters + +**Goal:** Add end-user vault settings stored in encrypted vault metadata, giving users control over: recycle bin retention period (default 30 days), delete behavior (soft delete to bin vs hard delete), and file versioning defaults (max versions per file, version cooldown period). Settings UI in the web app with sensible defaults matching current hardcoded values. +**Requirements**: None (deferred items from Phases 13, 17) +**Depends on:** Phase 38 + +Plans: _not yet planned_ + --- _Roadmap created: 2026-03-07_ -_Last updated: 2026-03-30_ -_Total M1.1 phases: 18 (18-35 complete) | Concern resolution: 5 phases_ +_Last updated: 2026-03-31_ +_Total M1.1 phases: 18 (18-35 complete) | Concern resolution: 5 phases | Post-milestone: 4 phases (36-39)_ diff --git a/.planning/codebase/CONCERNS.md b/.planning/codebase/CONCERNS.md index ea3de2835c..9348ae055b 100644 --- a/.planning/codebase/CONCERNS.md +++ b/.planning/codebase/CONCERNS.md @@ -1,13 +1,13 @@ # Codebase Concerns -**Analysis Date:** 2026-03-29 +**Analysis Date:** 2026-03-30 ## Tech Debt **Orphaned IPNS records on file/folder deletion:** - Issue: When files or folders are deleted, their IPNS records and TEE republish enrollments are handled by `fireAndForgetUnenroll()` in `packages/sdk/src/client.ts`. The web app services correctly delegate to the SDK. However, SDK-based unenrollment is fire-and-forget with no persistence — if the browser tab closes before the API call completes, unenrollments are silently dropped. -- Files: `packages/sdk/src/client.ts:150-163`, `apps/web/src/services/delete.service.ts:23`, `apps/web/src/services/folder.service.ts:457` +- Files: `packages/sdk/src/client.ts:156-174`, `apps/web/src/services/folder.service.ts` - Impact: Orphaned IPNS records accumulate in the TEE republish schedule. Each orphan wastes TEE compute and delegated routing bandwidth every 6 hours. Capacity warnings trigger at 1000+ records. - Fix approach: Persist a local unenrollment queue to IndexedDB. Flush on next session start before loading folders. @@ -39,6 +39,20 @@ - Impact: Type safety gap around the authentication flow. Could mask breaking SDK changes. - Fix approach: Create typed wrappers for Web3Auth SDK interactions using `unknown` + type guards. +**Residual `console.warn` calls in SDK packages (not structured logger):** + +- Issue: 11 `console.warn` calls in `packages/sdk/src/client.ts` and isolated calls in `packages/sdk/src/bin/index.ts:193` and `packages/sdk-core/src/ipns/index.ts:193` use raw `console.warn` rather than going through a structured logger. Phase 28 structured logging covered the web app but not the SDK packages (which have no logger dependency by design — they are zero-dependency packages). +- Files: `packages/sdk/src/client.ts` (lines 139, 168, 460, 752, 763, 817, 1012, 1022, 1073, 1437, 1516), `packages/sdk/src/bin/index.ts:193`, `packages/sdk-core/src/ipns/index.ts:193` +- Impact: SDK warnings bypass the web app's Faro transport and won't appear in Grafana dashboards. Debugging SDK-level issues in staging/production requires reading raw browser console logs. +- Fix approach: SDK and SDK-core are zero-dependency packages — they should not import a logging library. An acceptable alternative is accepting a logger callback in `CipherBoxClientConfig` and routing internal warnings through it. Alternatively, document that SDK warnings remain on raw console and are outside Faro coverage. + +**Duplicate file upload path bypasses Web Worker encryption:** + +- Issue: When a dropped file has the same name as an existing file in the target folder, it enters the duplicate/replacement path in `apps/web/src/hooks/useDropUpload.ts:199-255`. This path uses `encryptFile()` from `file-crypto.service.ts` which runs synchronously on the main thread, not through the `EncryptionWorkerService` introduced in Phase 37. +- Files: `apps/web/src/hooks/useDropUpload.ts:203`, `apps/web/src/services/file-crypto.service.ts` +- Impact: Large duplicate files (up to 100 MB) block the main thread during encryption, causing UI jank. Inconsistent with the batch upload path which uses the Worker. +- Fix approach: Route the duplicate upload through `getEncryptionWorker().createEncryptFn()` instead of `encryptFile()`. The `file-crypto.service` can remain for testing purposes. + ## Known Bugs **No active known bugs identified in the current codebase.** @@ -51,7 +65,7 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Risk: `clearBytes()` / `.fill(0)` cannot guarantee sensitive key material is erased from V8 heap, JIT-compiled code, or GC intermediaries. - Files: `packages/crypto/src/utils/memory.ts`, `apps/web/src/stores/vault.store.ts`, `apps/web/src/stores/auth.store.ts`, `apps/web/src/stores/folder.store.ts` -- Current mitigation: The codebase consistently uses `.fill(0)` on key buffers during logout and store cleanup. The Rust side uses `zeroize` crate. +- Current mitigation: The codebase consistently uses `.fill(0)` on key buffers during logout and store cleanup. The Rust side uses `zeroize` crate. The encryption Web Worker (Phase 37) also calls `clearBytes(fileKey)` before transferring the buffer, preventing key material from lingering in Worker memory. - Recommendations: Inherent JavaScript limitation. Acceptable for browser context; desktop Rust code uses proper zeroization. **Web3Auth localStorage usage:** @@ -75,14 +89,21 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Current mitigation: Guarded by `NODE_ENV` check and requires knowing the secret. - Recommendations: Ensure `TEST_LOGIN_SECRET` is never set when a production environment is deployed. Add monitoring alert for staging usage. +**Grafana Faro telemetry scrub relies on key-name allow-list:** + +- Risk: The `SENSITIVE_KEYS` set in `apps/web/src/lib/faro.ts` scrubs known field names (e.g., `privateKey`, `fileKey`, `rootFolderKey`). Unknown or newly added field names containing key material would not be scrubbed. +- Files: `apps/web/src/lib/faro.ts:12-22` +- Current mitigation: A secondary heuristic scrubs any string value matching 64+ hex characters regardless of key name. Binary `ArrayBuffer` / `ArrayBufferView` values are always redacted. +- Recommendations: When adding new fields that hold key material, ensure they are added to `SENSITIVE_KEYS`. The hex pattern heuristic is a safety net, not the primary defence. + ## Performance Bottlenecks -**Full file content buffering for AES-GCM encryption:** +**Full file content buffering for AES-GCM encryption (duplicate upload path):** -- Problem: Files encrypted with GCM mode are fully loaded into memory. The 100 MB file size limit means up to 100 MB memory per concurrent upload. -- Files: `packages/crypto/src/aes/encrypt.ts` -- Cause: AES-256-GCM requires full content for authentication tag computation. -- Improvement path: AES-256-CTR streaming encryption already exists (`packages/crypto/src/aes/encrypt-ctr.ts`, `packages/crypto/src/aes/decrypt-ctr.ts`) and is used for media streaming. Extend CTR usage to all uploads for reduced memory pressure. +- Problem: The duplicate file upload path (`useDropUpload.ts` duplicate branch) uses `encryptFile()` which reads the entire file into memory on the main thread and uses AES-256-GCM (full-buffer). The batch upload path for new files uses the Worker and selects CTR for eligible media files automatically. +- Files: `apps/web/src/services/file-crypto.service.ts`, `apps/web/src/hooks/useDropUpload.ts:203` +- Cause: The replacement/duplicate flow was not updated in Phase 37. It also doesn't benefit from Worker offloading. +- Improvement path: Migrate the duplicate path to use `EncryptionWorkerService` and pass `encryptFn` to the staging upload. This removes the main-thread block and enables CTR for large media files on the duplicate path. **IPNS polling for sync (30-second interval):** @@ -105,18 +126,18 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Files: `crates/fuse/src/lib.rs` (766 lines), `crates/fuse/src/write_ops.rs` (976 lines), `crates/fuse/src/read_ops.rs` (928 lines), `apps/desktop/src-tauri/vendor/fuser/src/channel.rs` - Why fragile: FUSE-T is a userspace NFS/SMB translation layer, not kernel FUSE. Numerous workarounds for macOS-specific issues (SMB opendir requires non-zero fh, rename truncates filenames by 8 bytes, UID mismatch under SMB proxy). Each macOS update could introduce new kernel-side behavior changes. - Safe modification: Always test with Finder, Terminal `ls`/`mv`/`cp`, and multi-file operations. -- Test coverage: Desktop E2E shell scripts exercise basic operations. Rust inline tests cover inode table and cache. No unit tests for filesystem callback implementations. +- Test coverage: Desktop E2E shell scripts exercise basic operations. Rust inline tests cover inode table and cache. No unit tests for filesystem callback implementations (won't fix — Desktop E2E is the appropriate level). **Windows FUSE implementation (WinFSP):** - Files: `crates/fuse/src/platform/windows/write_ops.rs` (1008 lines), `crates/fuse/src/platform/windows/operations.rs` (602 lines), `crates/fuse/src/platform/windows/read_ops.rs` (464 lines), `crates/fuse/src/platform/windows/dir_ops.rs` (206 lines) - Why fragile: 2280 lines of platform-specific FUSE code. Uses WinFSP which has different semantics from macOS FUSE-T. - Safe modification: Test on actual Windows with Explorer, cmd, and PowerShell. -- Test coverage: Desktop E2E runs on Windows in CI. No unit tests for Windows FUSE operations. +- Test coverage: Desktop E2E runs on Windows in CI. No unit tests for Windows FUSE operations (won't fix). **Mutex `unwrap()` calls in FUSE production code:** -- Files: `crates/fuse/src/lib.rs:141`, `:179`, `:183`, `crates/fuse/src/platform/windows/read_ops.rs`, `crates/fuse/src/platform/windows/write_ops.rs` +- Files: `crates/fuse/src/lib.rs:141`, `:179`, `:183`; `crates/fuse/src/platform/windows/read_ops.rs` (7 occurrences); `crates/fuse/src/platform/windows/write_ops.rs` (9 occurrences); `crates/fuse/src/platform/windows/dir_ops.rs:27` - Why fragile: 19+ `lock().unwrap()` calls on `Mutex` objects in FUSE production code (not tests). If any background thread panics while holding a lock, subsequent lock attempts will panic with "poisoned mutex", crashing the filesystem thread and unmounting the drive. - Safe modification: Replace with `lock().unwrap_or_else(|p| p.into_inner())` for poison recovery, or propagate errors via `EIO`. - Test coverage: No tests exercise panic-during-lock scenarios. @@ -142,6 +163,13 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Safe modification: Test all auth flows (email, Google, wallet) end-to-end after any Web3Auth dependency update. - Test coverage: Auth flow tested via E2E. Web3Auth unit mocking is complex. +**Phala Cloud CVM deployment (single provider):** + +- Files: `apps/tee-worker/src/services/tee-keys.ts`, `apps/tee-worker/src/index.ts` +- Why fragile: Phase 35 migrated TEE to Phala Cloud CVM using the dstack SDK (`@phala/dstack-sdk`). There is no fallback TEE provider — the previous AWS Nitro fallback option was not implemented. The dstack SDK is dynamically imported only inside `TEE_MODE=cvm`, making it unavailable for local testing without a CVM. +- Safe modification: Key derivation and signing logic is tested via unit tests with `TEE_MODE=test`. Production CVM changes require Phala console deployment. +- Test coverage: `apps/tee-worker/src/__tests__/tee-keys.test.ts` covers the test-mode derivation path. The CVM code path itself is not unit-testable outside a real Phala CVM. + ## Scaling Limits **IPNS record propagation and TEE republishing:** @@ -156,11 +184,11 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Limit: With FilePointers (~100 bytes each), a 1000-file folder produces ~100 KB of metadata before encryption. - Scaling path: This limit is enforced by design. For larger collections, users must create subfolders. -**File size limit (100 MB with GCM):** +**File size limit (100 MB):** - Current capacity: 100 MB per file per PRD constraint. -- Limit: Browser memory pressure with full-file buffering for GCM encryption. -- Scaling path: CTR streaming encryption is implemented but not yet the default for uploads. +- Limit: New file uploads (batch path) use the Worker and select CTR for eligible media files, reducing main-thread pressure. Duplicate uploads still buffer on the main thread (see Performance Bottlenecks). +- Scaling path: Migrate duplicate upload path to Worker + CTR (see Tech Debt). **Single Kubo IPFS node:** @@ -194,6 +222,12 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Impact: macOS updates can break FUSE-T. The NFS-to-SMB backend switch was forced by a macOS Sequoia kernel bug. - Migration plan: Monitor FUSE-T releases. Consider FileProvider API on macOS as long-term alternative. +**@phala/dstack-sdk:** + +- Risk: Phala-specific SDK for CVM key derivation. Tightly coupled to Phala Cloud infrastructure. No alternative provider is implemented. +- Impact: If Phala Cloud has an outage or breaking API change, the TEE republishing pipeline stops. No fallback means IPNS records eventually expire (48-hour TTL). +- Migration plan: Abstract key derivation behind an interface so alternative providers (AWS Nitro Enclaves, Azure Confidential Computing) can be plugged in. Currently blocked by the complexity of re-implementing key derivation + epoch management for a second provider. + ## Missing Critical Features **No offline support (web or desktop):** @@ -203,22 +237,16 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 ## Test Coverage Gaps -**Web app has minimal unit tests (3 test files for 165 source files):** +**Web app has minimal unit tests (won't fix):** -- What's not tested: All React components, most hooks, all services except sync store. -- Files: Only 3 test files exist: - - `apps/web/src/stores/__tests__/sync-store.test.ts` - - `apps/web/src/stores/__tests__/upload-error-recovery.test.ts` - - `apps/web/src/stores/__tests__/logout-security.test.ts` -- Risk: Regressions in folder operations, file uploads, auth flows, sharing, and bin operations go undetected until E2E tests or manual testing. The 1059-line `folder.service.ts`, 971-line `bin.service.ts`, and 791-line `SharedFileBrowser.tsx` have zero unit test coverage. -- Priority: High. Focus first on services (`folder.service.ts`, `bin.service.ts`, `share.service.ts`) and critical hooks (`useAuth.ts`, `useFolderMutations.ts`, `useFileOperations.ts`). +- Status: Won't fix. The web app layer is primarily thin wrappers around `@cipherbox/sdk` and React UI components. The 14 Playwright E2E suites in `tests/web-e2e/` cover all critical user flows end-to-end (upload, download, sharing, bin, search, auth, media preview, etc.). Unit testing these wrappers would duplicate E2E coverage with high mocking overhead and low marginal value. The SDK and SDK-core packages — where the actual logic lives — have comprehensive unit test suites (13 files each). -**TEE worker has minimal tests (1 test file for 13 source files):** +**TEE worker has improved but incomplete test coverage (5 test files for 14 source files):** -- What's not tested: IPNS signing, key management, epoch rotation, auth middleware, republish route handler. Only SSRF validation is tested. -- Files: `tee-worker/src/__tests__/ssrf-validation.test.ts` (single test file) -- Risk: Security-critical code (TEE key derivation, IPNS record signing) is largely untested. Regressions in epoch rotation or key decryption would silently break republishing. -- Priority: High. The TEE worker handles decrypted IPNS private keys — correctness is security-critical. +- What's not tested: The `migrate.ts` route handler, the `metrics.ts` middleware/route, and the production CVM code path in `tee-keys.ts`. +- Files: Tests at `apps/tee-worker/src/__tests__/` cover ssrf-validation, key-manager, auth middleware, tee-keys (test-mode only), and republish route. The `migrate.ts` route and `migration-worker.ts` service have no test coverage. +- Risk: The migration route handles epoch key rotation — a critical security operation. Bugs would go undetected until staging or production. The `migration-worker.ts` service is 19+ functions with no tests. +- Priority: High for migration-worker. Medium for metrics route. **Desktop app has no TypeScript unit tests:** @@ -227,12 +255,11 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Risk: Auth flow regressions, IPC communication errors. - Priority: Medium. Desktop E2E covers the critical paths. -**FUSE write operations have no unit tests:** +**FUSE write operations have no unit tests (won't fix):** -- What's not tested: All write operation implementations (create file, write data, rename, delete, mkdir, publish coordination) in both macOS and Windows variants. +- What's not tested: Write operation implementations (create file, write data, rename, delete, mkdir, publish coordination) in both macOS and Windows variants. - Files: `crates/fuse/src/write_ops.rs` (976 lines), `crates/fuse/src/platform/windows/write_ops.rs` (1008 lines) -- Risk: FUSE bugs cause data loss or mount crashes. Desktop E2E shell scripts cover basic flows but cannot exercise edge cases. -- Priority: High. At minimum, add unit tests for publish coordination logic and conflict merge behavior. +- Status: Won't fix. FUSE callbacks are thin plumbing between OS filesystem calls and the Rust SDK — unit testing them would require mocking the entire host OS filesystem layer, which is unreliable and brittle. These code paths are exercised by the Desktop E2E test suite (`tests/desktop-e2e/`) which tests actual file operations through the mounted filesystem. **API Client package has minimal tests:** @@ -241,6 +268,13 @@ Previous known bugs (upload modal stuck, auth refresh race) were fixed in PRs #5 - Risk: Low — mostly generated code. The real validation happens through SDK E2E tests that exercise the client. - Priority: Low. +**`useDropUpload` hook has no unit tests:** + +- What's not tested: The primary file drop handler including batch upload orchestration, duplicate detection, orphan CID cleanup, and quota check interactions. +- Files: `apps/web/src/hooks/useDropUpload.ts` (283 lines) +- Risk: Phase 37 significantly rewrote this hook to use `client.uploadFiles()` with Worker encryption. The dual-path logic (new files via SDK batch vs. duplicates via legacy encrypt+upload) has no coverage. A regression could silently break file uploads. +- Priority: High. This is the primary upload entry point in the web app. + --- - + diff --git a/.planning/codebase/TESTING.md b/.planning/codebase/TESTING.md index 2090dbd167..77b05b24e9 100644 --- a/.planning/codebase/TESTING.md +++ b/.planning/codebase/TESTING.md @@ -1,6 +1,6 @@ # Testing Patterns -**Analysis Date:** 2026-03-27 +**Analysis Date:** 2026-03-30 ## Test Framework @@ -68,19 +68,19 @@ cargo test -p cipherbox-crypto --test cross_language --no-default-features # Cr tests/ web-e2e/ playwright.config.ts # 3 webServers: mock-ipns, API, web - tests/ # Playwright spec files + tests/ # 14 Playwright spec files page-objects/ # Page Object Model classes dialogs/ # Dialog-specific POs file-browser/ # File browser POs pages/ # Full page POs utils/ # Test helpers (wallet-login, test-files, etc.) - fixtures/files/ # Binary test fixtures + fixtures/files/ # Binary test fixtures (PDF, MP4, MP3, small MP4) sdk-e2e/ vitest.config.ts # 120s timeout, sequential src/fixtures/test-harness.ts # Account provisioning + cleanup src/fixtures/multi-account.ts # Multi-user fixture for sharing tests src/helpers/ # Assertion + data generator helpers - src/suites/ # Test suites + src/suites/ # 11 test suites load/ vitest.config.ts # 600s timeout (10 min), sequential src/harness/ # Client pool, metrics, reporter, thresholds @@ -293,6 +293,20 @@ vi.mock('@cipherbox/sdk-core', () => ({ })); ``` +**Partial module mocking** (`importOriginal`) is used in `packages/sdk/src/__tests__/upload-batch.test.ts` to preserve non-mocked exports: + +```typescript +vi.mock('@cipherbox/sdk-core', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + loadFolderMetadata: vi.fn(), + uploadFile: vi.fn(), + // ... other specific overrides + }; +}); +``` + ### Web E2E (Playwright) **No mocking of backend.** Tests run against a real API + Postgres + IPFS + Redis stack. Authentication is mocked via: @@ -392,13 +406,17 @@ Downloads coverage artifacts from latest successful CI run, re-uploads tagged to - **API:** 40 `.spec.ts` files covering all services, controllers, guards, strategies, and pipes. Uses NestJS testing module with mocked repositories. - **Crypto:** 7 test files covering AES-GCM, AES-CTR, ECIES, Ed25519, HKDF, key hierarchy, vault IPNS. - **Core:** 10 test files covering folder metadata, IPNS records, vault blob, bin metadata, file IPNS, registry. -- **SDK-Core:** 11 test files covering download, folder, IPFS, IPNS, performance, pinning (5 providers), upload, vault. -- **SDK:** 8 test files covering client operations, bin operations, events, share operations, shared-write, pinning, upload concurrency. +- **SDK-Core:** 13 test files covering download, encryption-mode, folder, IPFS, IPNS, performance, pinning (5 providers), upload, vault. `upload.test.ts` covers `uploadFile()` including `encryptFn` injection, buffer-detachment safety, and `teeKeys` propagation. +- **SDK:** 13 test files covering client operations, bin operations, context, error handling, events, integration, key-cache, share operations, shared-write, pinning, upload concurrency, and batch upload. `upload-batch.test.ts` (Phase 37) covers the `uploadFiles()` orchestration: p-limit concurrency pool, single-publish, partial failure, per-file callbacks, event emission, key cleanup, BYO pinFn, and share re-wrap. - **API Client:** 1 test file (`instance.test.ts`). - **Web:** 3 test files (sync store, upload error recovery, logout security). - **TEE Worker:** 1 test file (`ssrf-validation.test.ts`). - **Rust (inline):** 19 Rust source files with `#[cfg(test)]` modules containing 158+ tests total across crypto, core, fuse, and sdk crates. +**Coverage intentional gaps:** + +- FUSE write operations (`crates/fuse/`) have no unit tests by design — covered by Desktop E2E instead. This is a documented won't-fix. + ### Integration Tests - **API IPNS:** `apps/api/src/ipns/__tests__/ipns.integration.spec.ts` (502 lines) -- tests IPNS service composition @@ -407,38 +425,44 @@ Downloads coverage artifacts from latest successful CI run, re-uploads tagged to ### SDK E2E Tests -Full API-backed tests running against real Postgres + IPFS + Redis: - -| Suite | File | Coverage | -| ---------------- | -------------------------------------------------------- | -------------------------------------------- | -| Vault Lifecycle | `tests/sdk-e2e/src/suites/vault-lifecycle.test.ts` | Init, duplicate, get, export, config, quota | -| Folder CRUD | `tests/sdk-e2e/src/suites/folder-crud.test.ts` | Create, rename, move, delete folders | -| File Operations | `tests/sdk-e2e/src/suites/file-operations.test.ts` | Upload, download, rename, move, delete files | -| Data Integrity | `tests/sdk-e2e/src/suites/data-integrity.test.ts` | Roundtrip verification, content checksums | -| IPNS Consistency | `tests/sdk-e2e/src/suites/ipns-consistency.test.ts` | IPNS publish/resolve, sequence numbers | -| Error Cases | `tests/sdk-e2e/src/suites/error-cases.test.ts` | Invalid inputs, 404s, auth failures | -| Concurrent Ops | `tests/sdk-e2e/src/suites/concurrent-operations.test.ts` | Parallel uploads, race conditions | -| Bin Operations | `tests/sdk-e2e/src/suites/bin-operations.test.ts` | Soft delete, restore, permanent delete | -| Share Operations | `tests/sdk-e2e/src/suites/share-operations.test.ts` | Create share, accept, revoke (multi-account) | -| Invite Link | `tests/sdk-e2e/src/suites/invite-link.test.ts` | Invite link creation and claiming | +Full API-backed tests running against real Postgres + IPFS + Redis. 11 suites total. + +| Suite | File | Coverage | +| ---------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Vault Lifecycle | `tests/sdk-e2e/src/suites/vault-lifecycle.test.ts` | Init, duplicate, get, export, config, quota | +| Folder CRUD | `tests/sdk-e2e/src/suites/folder-crud.test.ts` | Create, rename, move, delete folders | +| File Operations | `tests/sdk-e2e/src/suites/file-operations.test.ts` | Upload, download, rename, move, delete files | +| Data Integrity | `tests/sdk-e2e/src/suites/data-integrity.test.ts` | Roundtrip verification, content checksums | +| IPNS Consistency | `tests/sdk-e2e/src/suites/ipns-consistency.test.ts` | IPNS publish/resolve, sequence numbers | +| Error Cases | `tests/sdk-e2e/src/suites/error-cases.test.ts` | Invalid inputs, 404s, auth failures | +| Concurrent Ops | `tests/sdk-e2e/src/suites/concurrent-operations.test.ts` | Parallel uploads, race conditions | +| Bin Operations | `tests/sdk-e2e/src/suites/bin-operations.test.ts` | Soft delete, restore, permanent delete | +| Share Operations | `tests/sdk-e2e/src/suites/share-operations.test.ts` | Create share, accept, revoke (multi-account) | +| Invite Link | `tests/sdk-e2e/src/suites/invite-link.test.ts` | Invite link creation and claiming | +| Batch Upload | `tests/sdk-e2e/src/suites/batch-upload.test.ts` | `uploadFiles()` batch: 3-file batch, mixed sizes, progress callbacks, `files:batchUploaded` event | **Config:** 120s test timeout, 60s hook timeout, sequential execution, no file parallelism. ### Web E2E Tests (Playwright) -| Spec | File | Coverage | -| ------------------ | -------------------------------------------------- | ------------------------------------------------------------------------------------- | -| Full Workflow | `tests/web-e2e/tests/full-workflow.spec.ts` | Login, folder hierarchy, upload 12+ files, batch actions, move, edit, rename, cleanup | -| Recycle Bin | `tests/web-e2e/tests/recycle-bin.spec.ts` | Soft delete, restore, permanent delete, empty bin | -| Recovery | `tests/web-e2e/tests/recovery.spec.ts` | Account recovery with browser-based IPFS | -| MFA Flows | `tests/web-e2e/tests/mfa-flows.spec.ts` | MFA enrollment, device approval | -| Wallet Login | `tests/web-e2e/tests/wallet-login.spec.ts` | Ethereum wallet authentication | -| Sharing | `tests/web-e2e/tests/sharing-workflow.spec.ts` | Share folder, accept share, view shared items | -| Writable Shares | `tests/web-e2e/tests/writable-shares.spec.ts` | Write permissions, recipient uploads, permission changes | -| Invite Link | `tests/web-e2e/tests/invite-link-workflow.spec.ts` | Invite link creation and claiming | -| Search | `tests/web-e2e/tests/search-workflow.spec.ts` | Client-side search | -| Conflict Detection | `tests/web-e2e/tests/conflict-detection.spec.ts` | Concurrent edit conflict detection | -| Journey Timing | `tests/web-e2e/tests/journey-timing.spec.ts` | Performance timing metrics | +14 suites total. All use `test.describe.serial` and manage their own browser context + account lifecycle. + +| Spec | File | Coverage | +| ------------------ | -------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| Full Workflow | `tests/web-e2e/tests/full-workflow.spec.ts` | Login, folder hierarchy, upload 12+ files, batch actions, move, edit, rename, cleanup | +| Recycle Bin | `tests/web-e2e/tests/recycle-bin.spec.ts` | Soft delete, restore, permanent delete, empty bin | +| Recovery | `tests/web-e2e/tests/recovery.spec.ts` | Account recovery with browser-based IPFS | +| MFA Flows | `tests/web-e2e/tests/mfa-flows.spec.ts` | MFA enrollment, device approval | +| Wallet Login | `tests/web-e2e/tests/wallet-login.spec.ts` | Ethereum wallet authentication | +| Sharing | `tests/web-e2e/tests/sharing-workflow.spec.ts` | Share folder, accept share, view shared items | +| Writable Shares | `tests/web-e2e/tests/writable-shares.spec.ts` | Write permissions, recipient uploads, permission changes | +| Invite Link | `tests/web-e2e/tests/invite-link-workflow.spec.ts` | Invite link creation and claiming | +| Search | `tests/web-e2e/tests/search-workflow.spec.ts` | Client-side search | +| Conflict Detection | `tests/web-e2e/tests/conflict-detection.spec.ts` | Concurrent edit conflict detection | +| Journey Timing | `tests/web-e2e/tests/journey-timing.spec.ts` | Performance timing metrics | +| Batch Download | `tests/web-e2e/tests/batch-download.spec.ts` | Multi-select, SelectionActionBar download button, batch context menu (Phase 34) | +| Media Preview | `tests/web-e2e/tests/media-preview.spec.ts` | PDF canvas viewer, video player, audio player, corrupt file error state (Phase 34) | +| AES-CTR Streaming | `tests/web-e2e/tests/streaming-playback.spec.ts` | Large video CTR mode via service worker, small video GCM blob URL, decrypt badge (Phase 34) | **Playwright Config highlights:** @@ -447,6 +471,14 @@ Full API-backed tests running against real Postgres + IPFS + Redis: - No retries (`retries: 0` -- fix flakiness immediately) - 3 web servers auto-started: mock-ipns-routing (port 3001), API (port 3000), web app (port 5173) - Artifacts: screenshots, video, traces on failure only +- Suite-level timeouts set via `test.setTimeout()` in `beforeAll` (90-180s depending on suite) + +**Media test fixtures** (`tests/web-e2e/fixtures/files/`): + +- `test-document.pdf` -- PDF for preview tests +- `test-video.mp4` -- Large video (>256KB) for AES-CTR streaming path +- `test-video-small.mp4` -- Small video (<256KB) for AES-GCM blob URL path +- `test-audio.mp3` -- Audio for audio player preview ### Desktop E2E Tests @@ -460,6 +492,8 @@ Shell-script-based test suite orchestrated by `tests/desktop-e2e/scripts/run-all Runs on macOS (FUSE-T/SMB), Linux (FUSE3), Windows (WinFSP). Windows variant uses `run-all.ps1`. +**Note:** FUSE write operations have no unit tests by design. Desktop E2E is the sole test coverage for this layer. + ### Load Tests | Scenario | File | Description | @@ -524,6 +558,33 @@ afterAll(async () => { }); ``` +### Batch Upload Unit Test Pattern + +Tests for `uploadFiles()` use `setupBatchMocks()` + `setupFolder()` helpers and a `makeUploadResult()` factory: + +```typescript +describe('CipherBoxClient.uploadFiles - batch upload orchestration', () => { + let client: CipherBoxClient; + + beforeEach(() => { + vi.clearAllMocks(); + client = new CipherBoxClient(createTestConfig()); + }); + + it('publishes only successful files on partial failure (D-09)', async () => { + setupFolder(client); + setupBatchMocks(5, [1, 3]); // files at index 1 and 3 fail + vi.mocked(sdkCore.loadFolderMetadata).mockResolvedValue(null); + + const result = await client.uploadFiles('folder-ipns', makeTestFiles(5)); + + expect(result.successes).toHaveLength(3); + expect(result.failures).toHaveLength(2); + expect(sdkCore.updateFolderMetadataAndPublish).toHaveBeenCalledTimes(1); + }); +}); +``` + ### Page Object Model (Playwright) ```typescript @@ -541,4 +602,4 @@ export class FileListPage { --- - + diff --git a/.planning/todos/pending/2026-02-07-web-worker-large-file-encryption.md b/.planning/todos/done/2026-02-07-web-worker-large-file-encryption.md similarity index 100% rename from .planning/todos/pending/2026-02-07-web-worker-large-file-encryption.md rename to .planning/todos/done/2026-02-07-web-worker-large-file-encryption.md diff --git a/README.md b/README.md index 275027eb53..f5fbc0a628 100644 --- a/README.md +++ b/README.md @@ -16,34 +16,31 @@ This project is inspired by discussions and planning while working on [ChainSafe ## Features -- **Authentication** — Web3Auth MPC Core Kit: email OTP, Google OAuth, magic link, external wallet. MFA via device factor. Same user identity = same vault. -- **Encryption** — Client-side AES-256-GCM for files and metadata. ECIES secp256k1 key wrapping (per-file, per-folder random keys). Streaming AES-256-CTR for large media files. -- **File Management** — Upload, download, rename, move, delete. Nested folders up to 20 levels. Drag-and-drop. -- **Sync** — Multi-device via IPNS polling (~30s). Conflict detection and resolution. -- **Desktop** — macOS via Tauri v2 + FUSE-T virtual filesystem mount. Background sync with system tray. +- **Authentication** — Web3Auth MPC Core Kit: email OTP, Google OAuth, magic link, external wallet (SIWE). MFA via device factor with BIP39 recovery phrase. +- **Encryption** — Client-side AES-256-GCM for files and metadata. ECIES secp256k1 key wrapping (per-file, per-folder random keys). AES-256-CTR streaming for large media files. Web Worker offloading for batch encryption. +- **File Management** — Upload, download, rename, move, delete. Batch upload pipeline with concurrent encryption and pinning. Nested folders up to 20 levels. Drag-and-drop. File versioning with point-in-time restore (up to 10 versions per file). +- **Sharing** — User-to-user sharing with ECIES re-wrapping (read-only and read-write). Invite-link sharing with time-limited access tokens. Multi-recipient sharing. Lazy key rotation on revoke. +- **Recycle Bin** — 30-day soft-delete with restore and permanent delete. +- **Search** — Client-side encrypted search index across file and folder names. Fuzzy matching with Cmd/Ctrl+K shortcut. +- **Media Preview** — Image, PDF, video (streaming CTR decryption), and audio preview. +- **Sync** — Multi-device via IPNS polling (~30s). Conflict detection and resolution (sequence-number based). +- **Desktop** — macOS, Linux, and Windows via Tauri v2 + FUSE-T/libfuse3/WinFsp virtual filesystem mount. Background sync with system tray. SMB backend on macOS. - **TEE Republishing** — Automatic IPNS record refresh every 6 hours via Phala Cloud. Zero-knowledge: keys decrypted only inside hardware enclaves. - **Data Portability** — Full vault export (JSON + encrypted blobs). Standalone recovery with private key — no CipherBox required. - -- **Sharing** — User-to-user sharing with ECIES re-wrapping. Invite-link sharing with time-limited access tokens. -- **Recycle Bin** — 30-day soft-delete with restore. - -### Planned (v1.1+) - -- **Search** — Client-side encrypted search index across file and folder names. -- **Versioning** — File history with point-in-time restore. -- **Desktop (Windows & Linux)** — WinFSP / libfuse support. +- **Observability** — Structured logging (web), Prometheus metrics endpoint (API), Grafana Faro integration, load testing infrastructure. +- **BYO IPFS** — Bring-your-own IPFS node support via server-relay flow. Dual-pin mode with automatic migration. ## Architecture Overview ```text ┌──────────────┐ ┌──────────────┐ ┌───────────────┐ -│ Web / Desktop│────▶│ CipherBox │────▶│ PostgreSQL │ +│ Web / Desktop│────>│ CipherBox │────>│ PostgreSQL │ │ (Client) │ │ API │ │ (Metadata) │ └──────┬───────┘ └──────┬───────┘ └───────────────┘ │ │ │ Encrypted │ Relay │ blobs │ - ▼ ▼ + v v ┌──────────────┐ ┌───────────────┐ │ IPFS │ │ TEE Worker │ │ (Storage) │ │ (IPNS Refresh)│ @@ -54,17 +51,18 @@ The client encrypts everything locally. The API is a zero-knowledge relay — it ## Tech Stack -| Component | Technology | -| :----------------- | :-------------------------------------------- | -| **Frontend** | React 18 + TypeScript + Tailwind CSS | -| **Backend** | Node.js + NestJS + TypeScript | -| **Database** | PostgreSQL 16 | -| **Job Queue** | BullMQ + Redis | -| **Key Derivation** | Web3Auth MPC Core Kit | -| **Storage** | IPFS via Kubo | -| **Desktop** | Tauri v2 + FUSE-T / WinFSP | -| **TEE** | Phala Cloud (IPNS republishing) | -| **Crypto** | Web Crypto API (AES-256-GCM, ECIES secp256k1) | +| Component | Technology | +| :----------------- | :-------------------------------------------------------------- | +| **Frontend** | React 18 + TypeScript + Tailwind CSS | +| **Backend** | Node.js + NestJS + TypeScript | +| **Database** | PostgreSQL 16 | +| **Job Queue** | BullMQ + Redis | +| **Key Derivation** | Web3Auth MPC Core Kit | +| **Storage** | IPFS via Kubo | +| **Desktop** | Tauri v2 + FUSE-T (macOS) / libfuse3 (Linux) / WinFsp (Windows) | +| **TEE** | Phala Cloud (IPNS republishing) | +| **Crypto** | Web Crypto API (AES-256-GCM/CTR) + eciesjs (ECIES secp256k1) | +| **Observability** | Grafana Faro (web), Prometheus (API) | ## Project Structure @@ -73,15 +71,27 @@ cipher-box/ ├── apps/ │ ├── api/ # NestJS backend │ ├── web/ # React frontend -│ └── desktop/ # Tauri v2 desktop app +│ ├── desktop/ # Tauri v2 desktop app +│ └── tee-worker/ # Phala Cloud TEE worker ├── packages/ -│ ├── crypto/ # Shared encryption library -│ └── api-client/ # OpenAPI spec; generated typed client lives in apps/web/src/api/ -├── tee-worker/ # Phala Cloud TEE worker +│ ├── core/ # Shared TypeScript types and metadata schemas +│ ├── crypto/ # Shared encryption library (AES, ECIES, key derivation) +│ ├── sdk-core/ # Stateless SDK operations (encrypt, pin, folder ops) +│ ├── sdk/ # Stateful SDK client (CipherBoxClient) +│ └── api-client/ # Generated OpenAPI typed client +├── crates/ +│ ├── core/ # Shared Rust types and metadata schemas +│ ├── crypto/ # Rust encryption library +│ ├── sdk/ # Rust SDK (stateful orchestration, sync daemon) +│ ├── fuse/ # FUSE filesystem implementation (macOS + Windows) +│ └── api-client/ # Rust API client ├── tests/ -│ ├── e2e/ # Playwright E2E tests -│ └── desktop-e2e/ # Desktop E2E tests -└── docker/ # Docker Compose (PostgreSQL, IPFS, Redis) +│ ├── web-e2e/ # Playwright E2E tests (14 suites) +│ ├── sdk-e2e/ # SDK integration tests (11 suites) +│ ├── desktop-e2e/ # Desktop E2E tests +│ ├── load/ # Load and performance tests +│ └── vectors/ # Crypto test vectors +└── docs/ # Architecture, development, and specification docs ``` ## Getting Started @@ -96,15 +106,17 @@ See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for full setup instructions inclu ## Documentation -| Document | Description | -| :------------------------------------------------------------------------- | :----------------------------------------------------------- | -| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Encryption hierarchy, key derivation, threat model | -| [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Local setup, running, testing | -| [docs/AUTHENTICATION_ARCHITECTURE.md](docs/AUTHENTICATION_ARCHITECTURE.md) | Auth flow details | -| [docs/METADATA_SCHEMAS.md](docs/METADATA_SCHEMAS.md) | All metadata object schemas | -| [docs/VAULT_EXPORT_FORMAT.md](docs/VAULT_EXPORT_FORMAT.md) | Export/recovery data format | -| [docs/DATABASE_EVOLUTION_PROTOCOL.md](docs/DATABASE_EVOLUTION_PROTOCOL.md) | Migration discipline | -| [00-Preliminary-R&D/Documentation/](00-Preliminary-R&D/Documentation/) | Frozen v1 specifications (PRD, API, Data Flows, Client Spec) | +| Document | Description | +| :------------------------------------------------------------------------- | :------------------------------------------------- | +| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Encryption hierarchy, key derivation, threat model | +| [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) | Local setup, running, testing | +| [docs/AUTHENTICATION_ARCHITECTURE.md](docs/AUTHENTICATION_ARCHITECTURE.md) | Auth flow details | +| [docs/METADATA_SCHEMAS.md](docs/METADATA_SCHEMAS.md) | All metadata object schemas | +| [docs/FILESYSTEM_SPECIFICATION.md](docs/FILESYSTEM_SPECIFICATION.md) | Filesystem rules, naming, and constraints | +| [docs/VAULT_EXPORT_FORMAT.md](docs/VAULT_EXPORT_FORMAT.md) | Export/recovery data format | +| [docs/DATABASE_EVOLUTION_PROTOCOL.md](docs/DATABASE_EVOLUTION_PROTOCOL.md) | Migration discipline | +| [docs/CAPACITY.md](docs/CAPACITY.md) | Storage quotas and limits | +| [tests/TESTING_STRATEGY.md](tests/TESTING_STRATEGY.md) | SDK E2E and load testing architecture | ## License diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 98d1c79e44..8dcda4bce9 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -15,7 +15,7 @@ User Device ← Vault Data ← PostgreSQL ↓ Encrypted Keys IPFS (Kubo) ← Encrypted Files ↑ -TEE (Phala; Nitro fallback planned) ← IPNS Republish (every 6h) +TEE (Phala Cloud CVM) ← IPNS Republish (every 6h) ``` **Key Properties:** @@ -258,7 +258,7 @@ Client holds: Private key (RAM only) IPNS records expire after ~24h → backend scheduler triggers republish every 6h Client encrypts ipnsPrivateKey with TEE public key (ECIES) TEE decrypts in hardware, signs, zeroes key immediately -Providers: Phala Cloud (current) / AWS Nitro (planned fallback) +Provider: Phala Cloud CVM (hardware-backed key derivation via dstack SDK) ``` ## User Journey Example @@ -274,7 +274,7 @@ Providers: Phala Cloud (current) / AWS Nitro (planned fallback) ## Further Reading -- [Frozen Specifications](../00-Preliminary-R&D/Documentation/) — PRD, Technical Architecture, API Specification, Data Flows, Client Specification +- [Frozen Specifications](../.planning/milestones/m0/Documentation/) — PRD, Technical Architecture, API Specification, Data Flows, Client Specification - [Authentication Architecture](AUTHENTICATION_ARCHITECTURE.md) — detailed auth flow documentation - [Metadata Schemas](METADATA_SCHEMAS.md) — all metadata object schemas with field tables - [Vault Export Format](VAULT_EXPORT_FORMAT.md) — export/recovery data format diff --git a/docs/DATABASE_EVOLUTION_PROTOCOL.md b/docs/DATABASE_EVOLUTION_PROTOCOL.md index 15f7fcaf7a..2a05def707 100644 --- a/docs/DATABASE_EVOLUTION_PROTOCOL.md +++ b/docs/DATABASE_EVOLUTION_PROTOCOL.md @@ -191,7 +191,7 @@ Format: `{prefix}{table_name}_{column(s) or description}` ## 6. Current Schema Reference -12 tables as of Phase 14 (version 0.15.0): +14 tables as of Phase 37: | # | Table | Entity File | Purpose | Foreign Keys | | --- | ------------------------- | ------------------------------------------- | ----------------------------------------- | ----------------------------------------- | @@ -207,6 +207,8 @@ Format: `{prefix}{table_name}_{column(s) or description}` | 10 | `shares` | `shares/entities/share.entity.ts` | User-to-user share grants | `sharer_id`, `recipient_id` -> `users.id` | | 11 | `share_keys` | `shares/entities/share-key.entity.ts` | Per-item encrypted keys for shares | `share_id` -> `shares.id` | | 12 | `device_approvals` | `device-approval/device-approval.entity.ts` | MFA device approval requests | -- (uses `user_id` varchar, not FK) | +| 13 | `share_invites` | `shares/entities/share-invite.entity.ts` | Invite link tokens for sharing | `sharer_id` -> `users.id` | +| 14 | `pin_migrations` | `migration/migration.entity.ts` | BYO-IPFS pin migration tracking | `user_id` -> `users.id` | All entity files are relative to `apps/api/src/`. @@ -214,16 +216,22 @@ All entity files are relative to `apps/api/src/`. ### Migration File Inventory -| Timestamp | File | Type | Tables Affected | -| --------------- | ------------------------------------ | ------------------ | ----------------------------------------- | -| `1700000000000` | `FullSchema.ts` | Baseline | 11 tables (all except `device_approvals`) | -| `1737520000000` | `MakeFolderIpnsTeeFieldsNullable.ts` | Alter column | `folder_ipns` | -| `1738972800000` | `AddTokenPrefix.ts` | Add column + index | `refresh_tokens` | -| `1739800000000` | `AddRecordTypeToFolderIpns.ts` | Add column | `folder_ipns` | -| `1740000000000` | `AddDeviceApprovals.ts` | Create table | `device_approvals` | -| `1740200000000` | `AddAuthMethodsUniqueConstraint.ts` | Replace index | `auth_methods` | -| `1740250000000` | `AddSharesTables.ts` | Create table | `shares`, `share_keys` | -| `1740300000000` | `SharesPartialUniqueIndex.ts` | Replace constraint | `shares` | +| Timestamp | File | Type | Tables Affected | +| --------------- | ------------------------------------ | ------------------ | ---------------------------------------------------------------------------- | +| `1700000000000` | `FullSchema.ts` | Baseline | 11 tables (all except `device_approvals`, `share_invites`, `pin_migrations`) | +| `1737520000000` | `MakeFolderIpnsTeeFieldsNullable.ts` | Alter column | `folder_ipns` | +| `1738972800000` | `AddTokenPrefix.ts` | Add column + index | `refresh_tokens` | +| `1739800000000` | `AddRecordTypeToFolderIpns.ts` | Add column | `folder_ipns` | +| `1740000000000` | `AddDeviceApprovals.ts` | Create table | `device_approvals` | +| `1740200000000` | `AddAuthMethodsUniqueConstraint.ts` | Replace index | `auth_methods` | +| `1740250000000` | `AddSharesTables.ts` | Create table | `shares`, `share_keys` | +| `1740300000000` | `SharesPartialUniqueIndex.ts` | Replace constraint | `shares` | +| `1740400000000` | `AddShareInvites.ts` | Create table | `share_invites` | +| `1740500000000` | `BackfillIpnsSequenceNumbers.ts` | Data migration | `folder_ipns` | +| `1740600000000` | `AddByoUserFlag.ts` | Add column | `vaults` | +| `1742000000000` | `AddPinMigrations.ts` | Create table | `pin_migrations` | +| `1743000000000` | `AddWritableShares.ts` | Add columns | `shares` | +| `1743100000000` | `WidenShareKeyType.ts` | Alter column | `share_keys` | --- @@ -347,6 +355,6 @@ During the creation of this protocol, the `device_approvals` table was identifie --- -_Protocol version: 1.0_ -_Last updated: 2026-02-22_ +_Protocol version: 1.1_ +_Last updated: 2026-03-31_ _Applies to: All database entities and migrations in `apps/api/src/`_ diff --git a/docs/FILESYSTEM_SPECIFICATION.md b/docs/FILESYSTEM_SPECIFICATION.md new file mode 100644 index 0000000000..c3a77b986c --- /dev/null +++ b/docs/FILESYSTEM_SPECIFICATION.md @@ -0,0 +1,191 @@ +# CipherBox Filesystem Specification + +This document defines the rules, constraints, and rationale governing the CipherBox virtual filesystem. These rules apply across all platforms (web, desktop macOS, desktop Linux, desktop Windows) and are enforced at different layers depending on the operation. + +## Design Principles + +CipherBox must present a consistent filesystem experience across three very different access surfaces: + +1. **Web browser** — drag-and-drop uploads, folder tree UI +2. **macOS desktop** — FUSE-T virtual mount via SMB backend, accessed through Finder and Terminal +3. **Linux desktop** — kernel FUSE (libfuse3) virtual mount, accessed through file manager and terminal +4. **Windows desktop** — WinFsp virtual mount, accessed through Explorer, PowerShell, and Git Bash + +The target invariant is that the strictest platform constraint (Windows) should govern all platforms, ensuring files created anywhere are accessible everywhere. In practice, several known gaps exist where the web layer does not yet enforce Windows-compatible rules — these are documented in the [Known Gaps](#known-gaps) section below. + +## Naming Rules + +### Case Handling + +| Platform | Storage | Lookup | Implementation | +| -------- | ---------- | -------------------------------- | ------------------------------------------------------------- | +| Web | As-entered | Case-sensitive (`===`) | `folder.service.ts`, `sdk-core/folder` | +| macOS | As-entered | NFC-normalized | `inode.rs:normalize_name()` via `unicode-normalization` crate | +| Linux | As-entered | NFC-normalized | Same `fuse` feature as macOS; case-sensitive | +| Windows | As-entered | Case-insensitive (lowercase key) | `inode.rs:normalize_name()` via `.to_lowercase()` | + +**Rationale:** Original casing is always preserved in the encrypted metadata (`InodeData.name` / `FolderChild.name`). The normalization only affects HashMap key lookups in the FUSE layer. On macOS, NFC normalization prevents mismatches between composed and decomposed Unicode forms (e.g., `e` + combining acute vs. precomposed `e`). On Windows, lowercase folding implements the case-insensitive semantics that Explorer and all Windows applications expect. + +**Current gap:** The web UI uses strict case-sensitive matching for duplicate detection. A user could create `Report.pdf` and `report.pdf` in the same folder on the web, but these would collide when the vault is mounted on Windows. This is a known limitation — the web layer does not currently enforce Windows-compatible case-insensitive uniqueness. + +### Character Restrictions + +| Character | Status | Rationale | +| -------------------------- | ------------- | ------------------------------------------------------------------------------ | +| Null byte (`\0`) | Forbidden | Invalid in all filesystem APIs; rejected by UTF-8 validation in FUSE callbacks | +| Full UTF-8 range | Allowed | Stored encrypted in metadata; no server-side interpretation | +| Emoji and CJK | Allowed | Full Unicode support across all platforms | +| Path separators (`/`, `\`) | Not validated | Names are stored in flat per-folder child arrays, not as paths | + +**Note:** Windows reserves additional characters (`<`, `>`, `:`, `"`, `|`, `?`, `*`) in filenames. CipherBox does not currently validate against these at the web or SDK layer. Files created with these characters on the web would fail to materialize on a Windows FUSE mount. This is a known gap. + +### Reserved Names + +CipherBox filters platform-specific system files that should never be created, synced, or displayed. These are silently rejected during file creation (EACCES) and hidden from directory listings in the FUSE layer. + +**Cross-platform filter** (`helpers.rs:is_platform_special()`): + +| Pattern | Platform | Purpose | +| ------------------------ | -------- | ----------------------------- | +| `._*` (prefix) | macOS | Resource fork files | +| `.DS_Store` | macOS | Finder metadata | +| `.Trashes` | macOS | Trash directory | +| `.fseventsd` | macOS | File system events | +| `.Spotlight-V100` | macOS | Spotlight index | +| `.hidden`, `.localized` | macOS | Finder display hints | +| `.metadata_never_index*` | macOS | Spotlight indexing hints | +| `.ql_disablecache` | macOS | Quick Look cache control | +| `.ql_disablethumbnails` | macOS | Quick Look thumbnail control | +| `DCIM` | macOS | Digital camera images dir | +| `Thumbs.db` | Windows | Explorer thumbnail cache | +| `desktop.ini` | Windows | Explorer folder customization | +| `.directory` | Linux | KDE directory metadata | +| `.Trash-*` (prefix) | Linux | Per-user trash directories | +| `.gvfs` | Linux | GNOME virtual filesystem | +| `.xdg-volume-info` | Linux | XDG volume information | + +**Windows-specific filter** (`helpers.rs:is_windows_special()`): + +| Pattern | Purpose | +| ------------------------------ | ------------------------------------ | +| `$recycle.bin` | Recycle bin directory | +| `System Volume Information` | NTFS system metadata | +| `Recycler` | Legacy recycle bin | +| `pagefile.sys`, `swapfile.sys` | Virtual memory files | +| `hiberfil.sys` | Hibernation file | +| `$*` (prefix) | NTFS alternate data streams | +| `*:Zone.Identifier*` | Downloaded file security zone marker | + +**Rationale:** These files are created automatically by operating systems and file managers. Syncing them wastes storage quota, creates noise in the file browser, and can cause errors when the vault is accessed from a different platform. Filtering at the FUSE layer (not the web layer) means the web UI has no awareness of these restrictions — files with these names can be uploaded via the web. They simply won't appear when the vault is mounted on desktop. + +**Not currently filtered:** Windows reserved device names (`CON`, `PRN`, `AUX`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9`). These would cause issues on Windows FUSE mounts but are unlikely to be created intentionally. + +## Size Limits + +### File Size + +| Limit | Value | Enforcement | Constant | +| ------------- | ------ | ------------------------------------------- | ------------------------------------------------------------------------------------- | +| Max file size | 100 MB | Client-side (web upload), server-side (API) | `MAX_FILE_SIZE` in `useDropUpload.ts`, `MaxFileSizeValidator` in `ipfs.controller.ts` | + +**Rationale:** 100 MB balances usability with memory constraints. AES-256-GCM requires the full plaintext in memory for authentication tag computation. Large media files use AES-256-CTR which supports streaming, but the 100 MB limit applies uniformly. The Web Worker encryption offload helps with CPU blocking but does not change the memory requirement. + +### Storage Quota + +| Limit | Value | Enforcement | Constant | +| ----------- | ------- | -------------------------------------------------- | ----------------------------------- | +| Total quota | 500 MiB | Server-side (`vault.service.ts`), client pre-check | `QUOTA_LIMIT_BYTES` / `QUOTA_BYTES` | + +**Rationale:** Technology demonstrator limit. Quota tracks encrypted blob sizes (not plaintext sizes). BYO-IPFS users bypass this limit (advisory only) since they manage their own storage. + +## Folder Structure + +### Depth Limit + +| Limit | Value | Enforcement | Constant | +| ---------------- | --------- | ---------------------------------------- | ------------------ | +| Max folder depth | 20 levels | Client-side (create folder, move folder) | `MAX_FOLDER_DEPTH` | + +**Rationale:** Each folder level requires a separate IPNS resolve + AES-256-GCM decrypt to traverse. Deep nesting creates latency during navigation and increases the number of IPNS records the TEE must republish. 20 levels is generous for practical use while bounding operational costs. + +**Enforcement points:** + +- `folder.service.ts:170` — `createFolder` checks `parentDepth >= MAX_FOLDER_DEPTH` +- `folder.service.ts:759` — `moveFolder` checks `destDepth + 1 + subtreeDepth > MAX_FOLDER_DEPTH` +- `useFolderMutations.ts:73` — duplicate check in React hooks layer + +### Duplicate Names + +Files and folders within the same parent must have unique names. Duplicate detection is case-sensitive at the web/SDK layer. + +**Enforcement:** Before any create, rename, or move operation, the target folder's children are scanned: + +```typescript +const nameExists = children.some((c) => c.name === newName && c.id !== childId); +if (nameExists) throw new Error('An item with this name already exists'); +``` + +**Cross-type collision:** A file cannot have the same name as a folder in the same parent. The web upload layer explicitly checks for file-to-folder name collisions before starting uploads. + +**Batch upload deduplication:** Within a single batch upload, duplicate filenames are rejected before any files are read into memory. + +## File Versioning + +| Limit | Value | Enforcement | Constant | +| ----------------- | ---------- | ------------------ | ----------------------- | +| Max versions/file | 10 | SDK and FUSE layer | `MAX_VERSIONS_PER_FILE` | +| Version cooldown | 15 minutes | Desktop FUSE only | `VERSION_COOLDOWN_MS` | + +**Rationale:** Version history is stored inside the file's IPNS metadata. Each version entry contains a CID, key, IV, and timestamp. Capping at 10 prevents metadata bloat. The 15-minute cooldown on desktop prevents rapid saves (e.g., auto-save in text editors) from consuming all version slots. + +**Web behavior:** The web UI creates a new version on every file replace operation. There is no cooldown — the user explicitly triggers each replace via the upload dialog. + +## FUSE Mount Specifics + +### Content Download + +| Parameter | Value | Constant | +| ---------------- | ------ | -------------------------- | +| Download timeout | 120s | `CONTENT_DOWNLOAD_TIMEOUT` | +| Block size | 4096 B | `BLOCK_SIZE` | + +**Rationale:** File content is fetched from IPFS via the Kubo RPC API on `open()` and cached in memory. Large files (up to 100 MB) can take 30-60 seconds over slow connections. The 120-second timeout accommodates this while preventing indefinite hangs. After the initial fetch, all `read()` calls are served from cache. + +### Mount Backend + +| Platform | Backend | Notes | +| -------- | ------- | --------------------------------------------------------------------------------------------------------- | +| macOS | SMB | FUSE-T's SMB backend. NFS backend has unfixable kernel bug (WRITE RPCs never reach FUSE-T for new files). | +| Linux | libfuse | Kernel FUSE via libfuse3. Standard FUSE semantics, no translation layer. | +| Windows | WinFsp | Native user-mode filesystem driver. | + +### Inode Management + +- **Root inode:** Always `1` (standard FUSE convention) +- **Inode stability:** `populate_folder` reuses existing inode numbers for same-name children. Allocating new inodes for unchanged files causes NFS "stale file handle" disconnects on macOS. +- **Lazy loading:** Folder children are populated on first `readdir`/`lookup`, not on mount. This avoids loading the entire vault tree upfront. + +## Encryption Modes + +| Mode | Usage | File size threshold | Notes | +| ----------- | ------------------------------------- | ---------------------- | ------------------------------------------------ | +| AES-256-GCM | Default for all files | Any | Authenticated encryption; full file in memory | +| AES-256-CTR | Large media files (encrypt + decrypt) | > 256 KB (video/audio) | Streaming encrypt/decrypt without full buffering | + +**Rationale:** GCM provides authentication (tamper detection) but requires the entire plaintext in memory. CTR mode is used for large media files (video, audio) both on upload and download, enabling streaming without buffering the entire file. The encryption mode is recorded in file metadata so the correct decryption path is selected on download. + +## Metadata Storage + +File and folder names, timestamps, and structural information are stored as encrypted JSON in IPNS records. The server never sees plaintext names or folder structure. See [METADATA_SCHEMAS.md](METADATA_SCHEMAS.md) for the full schema reference and [METADATA_EVOLUTION_PROTOCOL.md](METADATA_EVOLUTION_PROTOCOL.md) for change management rules. + +## Known Gaps + +These are documented limitations where cross-platform consistency is not yet enforced: + +1. **Web case-sensitivity vs. Windows case-insensitivity** — Files created via the web with names differing only by case will collide on Windows desktop mount. +2. **Windows reserved characters** — Characters like `<`, `>`, `:`, `"`, `|`, `?`, `*` are not validated at the web/SDK layer. Files with these characters will fail on Windows FUSE mounts. +3. **Windows reserved device names** — `CON`, `PRN`, `NUL`, `COM1`-`COM9`, `LPT1`-`LPT9` are not validated. +4. **Path length** — No total path length validation. Windows has a 260-character path limit (unless long path support is enabled). Deep nesting + long names could exceed this on Windows. +5. **Leading/trailing spaces and dots** — Windows silently strips these from filenames. Files with trailing dots/spaces created on the web would appear with different names on Windows. + +