docs: comprehensive documentation update for phases 28-37

[FSM1]

Summary

Comprehensive documentation audit and update covering all changes from phases 28-37 plus roadmap planning for phases 38-39.

New documentation

• docs/FILESYSTEM_SPECIFICATION.md — filesystem rules, naming constraints, case handling (per-platform including Linux), reserved names, size/quota/depth limits, encryption modes (GCM + CTR for large media), FUSE mount specifics, and known cross-platform gaps

Updated documentation

• README.md — updated features, project structure, tech stack (added Linux, observability, BYO IPFS), documentation links
• docs/ARCHITECTURE.md — TEE updated from "Nitro fallback planned" to Phala Cloud CVM; fixed frozen specs path
• docs/DATABASE_EVOLUTION_PROTOCOL.md — added 6 missing migrations, 2 new tables (share_invites, pin_migrations), updated from 12 to 14 tables
• CLAUDE.md — fixed stale 00-Preliminary-R&D/ paths, TEE description, republish interval

Planning updates

• .planning/DEFERRED.md — thorough audit removing 10+ implemented/irrelevant items (Grafana Faro, session replay, FUSE CTR streaming, Linux FUSE, vault v1 columns, batch pin warnings, per-file IPNS conflict detection, bin retention env var, DTS circular dep, useFileOperations decomposition); added Phase 37 upload pipeline deferrals
• .planning/FEATURES.md — added batch upload, batch download, media preview, streaming playback, Web Worker encryption, structured logging, planned vault settings; updated E2E test suites (14 web, 11 SDK)
• .planning/ROADMAP.md — added Phase 38 (retire deprecated web services + circular dep) and Phase 39 (user-configurable vault parameters: bin retention, delete behavior, versioning defaults)
• .planning/codebase/CONCERNS.md — marked FUSE unit tests as won't-fix (Desktop E2E is the appropriate test level)
• Moved Web Worker encryption todo from pending to done

Test plan

• Verify all doc links resolve (frozen specs path, cross-references)
• Spot-check DEFERRED.md items against codebase
• Verify migration inventory matches apps/api/src/migrations/
• Verify FEATURES.md matrix against actual test files in tests/

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Batch upload pipeline, file versioning (point-in-time restore), client-side search (Cmd/Ctrl+K), media preview (image/PDF/video/audio streaming), enhanced sharing (multi-recipient/time-limited links), Recycle Bin (30-day restore)
  • Expanded desktop support: macOS, Linux, Windows

• Documentation

  • Added cross-platform filesystem specification, updated README, roadmap (phases 36–39), architecture, testing, and database evolution docs

[coderabbitai]

Walkthrough

This PR updates planning and project documentation: expands deferred milestone scope, restructures deferred items, revises the feature/E2E matrix, extends the README and project layout, adds a cross-platform filesystem specification, appends roadmap phases, and updates codebase concerns and testing matrices.

Changes

Cohort / File(s)	Summary
Planning & Roadmap ​.planning/DEFERRED.md, ​.planning/FEATURES.md, ​.planning/ROADMAP.md	Expanded deferred inventory (moved v1.1 coverage to later phases, added Observability and Upload Pipeline deferred items, removed Web Worker encryption from deferred to implemented/Phase 37), updated FEATURES E2E matrix (batch upload/download, media/streaming, AES-CTR/Web Worker), and added post-v1.1 phases 36–39 with metadata updates.
Project Docs / README & Architecture README.md, docs/ARCHITECTURE.md	Expanded README with new capabilities (batch upload pipeline, streaming AES-CTR, versioning, recycle bin, sharing changes), updated tech-stack and repo layout (core/sdk/crates/apps), and switched TEE provider reference to Phala Cloud CVM.
Filesystem Spec (new) docs/FILESYSTEM_SPECIFICATION.md	Added cross-platform filesystem specification: platform name-normalization/lookup rules, reserved-name filters, filename character rules, operational limits (100 MiB/file, 500 MiB quota, 20 depth, 10 versions), FUSE/inode behavior, and encryption mode selection (GCM default, CTR for large media).
Codebase Concerns & Testing ​.planning/codebase/CONCERNS.md, ​.planning/codebase/TESTING.md	Added and refined tech-debt/security concerns (console.warn residues, duplicate-upload encrypt-on-main-thread, worker zeroing), marked FUSE write-op unit-tests as “won’t fix”, updated unit/E2E suite counts and batch-upload test patterns, and expanded TEE/fixture/test coverage notes.
Database / Migrations docs/DATABASE_EVOLUTION_PROTOCOL.md	Bumped schema to include share_invites and pin_migrations; added recent migration entries and updated protocol metadata to version 1.1 (2026-03-31).
Auxiliary planning files & misc ​.claude/claude.md, README.md (metadata sections)	Adjusted planning metadata/footer dates, migrated specification paths in claude doc, changed TEE republishing schedule/provider references, corrected minor typos, and reorganized documentation links/sections.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• docs: feature matrix, codebase refresh, E2E todos #380 — Overlaps edits to .planning/DEFERRED.md and .planning/FEATURES.md (milestone and feature inventory changes).
• docs: cleanup deferred items, security backlog, and add Phase 34 #391 — Related updates to structured-logging / observability entries referenced in the deferred inventory.
• docs: add Phase 19 someguy IPNS performance baselines #336 — Overlapping planning/doc edits touching the IPNS sidecar and pending-todo items.

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'docs: comprehensive documentation update for phases 28-37' accurately summarizes the main change—a broad documentation audit covering multiple planning phases with updates to architecture, roadmap, deferred items, features, and new filesystem specification.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/update-project-documentation

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)

README.md (2)

11-11: ⚠️ Potential issue | 🟠 Major

Auth identity statement is too strong and currently inaccurate.

“Same vault regardless of login method” implies automatic cross-method identity convergence. In practice, cross-method sharing requires explicit auth-method linking. Please narrow this wording.

Based on learnings, “There is no automatic convergence — cross-method identity sharing requires explicit user-initiated linking via the Settings page.”

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@README.md` at line 11, The sentence claiming “the same user always gets the
same vault regardless of login method” is too strong and inaccurate; update the
README text containing that phrase to clarify that there is no automatic
cross-method convergence and that cross-method identity sharing requires
explicit user-initiated linking (e.g., via the Settings page), replacing or
rewording the clause to reflect explicit linking is required.

---

50-50: ⚠️ Potential issue | 🟠 Major

Zero-knowledge architecture text conflicts with current server crypto-material stance.

Line 50 says the API “stores encrypted keys,” which conflicts with the current documented zero-crypto-material server model. Please update this sentence to reflect the post-migration state precisely.

Based on learnings, “As of Phase 20 … the server stores ZERO crypto material by design.”

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@README.md` at line 50, Update the README sentence that currently reads
"stores encrypted keys" under the "Zero-knowledge relay" description to reflect
the post-migration design: replace that phrase with a clear statement such as
"stores ZERO crypto material by design (as of Phase 20)" and ensure the
surrounding sentence still conveys that the client encrypts locally and the API
only routes IPFS/IPNS operations without holding any crypto material; keep the
existing link to ARCHITECTURE.md intact for full details.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/FILESYSTEM_SPECIFICATION.md`:
- Line 150: Update the sentence that says "staging IPFS gateway" to reflect that
file content is fetched via Kubo RPC API endpoints (not an IPFS gateway); e.g.,
change wording in the rationale paragraph that describes fetching on open() and
caching so it states "fetched from IPFS via Kubo RPC endpoints" and add a short
clause clarifying "the IPFS gateway is not used" while keeping the 120‑second
timeout rationale and the note that subsequent read() calls are served from
cache.
- Line 13: Reword the sentence beginning "The strictest platform constraint
wins." to state it as the target invariant (CipherBox aims to enforce
Windows-compatible filename rules across platforms) and immediately follow it
with a concise "Current deviations" subsection that enumerates the documented
exceptions (e.g., case-sensitive web duplicates, permitted Windows-invalid
characters in some contexts) and their remedial tracking status; update the
enforcement/constraint checklist language elsewhere (the sections referencing
case-sensitive web duplicates and Windows-invalid characters) to clearly mark
them as known deviations from the target invariant rather than contradictory
enforcement requirements so readers can distinguish the intended invariant from
current gaps.
- Around line 44-46: Update the document to reference the actual helper function
names (is_macos_reserved_name, is_windows_reserved_name,
is_system_reserved_name) instead of the incorrect
helpers.rs:is_platform_special() / is_windows_special(), and explicitly mark
each pattern row as “implemented” when covered by those functions or
“planned/superset” when not yet implemented; also update the “Cross-platform
filter” table header and the entries referenced in the later section (lines
~65-76) to use the same corrected symbol names and implementation status so the
docs match the codebase.

In `@README.md`:
- Line 64: Update the README table row that currently reads "Web Crypto API
(AES-256-GCM/CTR, ECIES secp256k1)" to clearly separate native Web Crypto usage
from library implementations: replace the cell content to mention Web Crypto API
for AES-256-GCM/CTR and HKDF-SHA256 and explicitly note eciesjs for ECIES
secp256k1 (or use the concise form "Web Crypto API (AES-256-GCM, AES-256-CTR,
HKDF-SHA256) + eciesjs"); edit the table row string that contains "Web Crypto
API (AES-256-GCM/CTR, ECIES secp256k1)" accordingly.

---

Outside diff comments:
In `@README.md`:
- Line 11: The sentence claiming “the same user always gets the same vault
regardless of login method” is too strong and inaccurate; update the README text
containing that phrase to clarify that there is no automatic cross-method
convergence and that cross-method identity sharing requires explicit
user-initiated linking (e.g., via the Settings page), replacing or rewording the
clause to reflect explicit linking is required.
- Line 50: Update the README sentence that currently reads "stores encrypted
keys" under the "Zero-knowledge relay" description to reflect the post-migration
design: replace that phrase with a clear statement such as "stores ZERO crypto
material by design (as of Phase 20)" and ensure the surrounding sentence still
conveys that the client encrypts locally and the API only routes IPFS/IPNS
operations without holding any crypto material; keep the existing link to
ARCHITECTURE.md intact for full details.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8dff0d4c-a99f-4773-9b67-3839c35b1cbf

📥 Commits

Reviewing files that changed from the base of the PR and between 1c175d6 and c5ecb54.

📒 Files selected for processing (4)

• .planning/DEFERRED.md
• .planning/FEATURES.md
• README.md
• docs/FILESYSTEM_SPECIFICATION.md

[coderabbitai]

Actionable comments posted: 2

♻️ Duplicate comments (1)

README.md (1)

64-64: ⚠️ Potential issue | 🟡 Minor

Separate Web Crypto primitives from ECIES library usage in the Crypto row.

Line 64 currently implies ECIES is provided by Web Crypto API. In this repo, ECIES secp256k1 is provided by eciesjs, while Web Crypto is used for AES/HKDF.

#!/bin/bash
# Verify docs vs implementation references for ECIES and Web Crypto usage.
rg -n -C2 'Web Crypto API|ECIES|eciesjs' README.md apps packages .github || true

Based on learnings: “Use Web Crypto API for AES-256-GCM, AES-256-CTR, and HKDF-SHA256. Use eciesjs for ECIES secp256k1 key wrapping.”

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@README.md` at line 64, The README's Crypto table conflates Web Crypto
primitives with the ECIES implementation; update the Crypto row so it clearly
separates Web Crypto API usage (AES-256-GCM, AES-256-CTR, HKDF-SHA256) from
ECIES (implemented via the eciesjs library, secp256k1). Edit the line that
currently reads "Web Crypto API (AES-256-GCM/CTR, ECIES secp256k1)" to instead
list Web Crypto primitives and explicitly state "ECIES (secp256k1) via eciesjs"
so readers can map implementation to libraries correctly.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.planning/codebase/CONCERNS.md:
- Around line 225-229: Update the outage impact wording in the `@phala/dstack-sdk`
section to state IPNS records expire after 48 hours (not 72); change the
"72-hour TTL" text to "48-hour TTL" so it matches the TEE signer configuration
constant TEE_RECORD_LIFETIME_MS and the actual signer behavior.

In @.planning/DEFERRED.md:
- Line 3: Update the `Last updated` metadata in .planning/DEFERRED.md to a
non-future audit date (change "Last updated: 2026-03-31" to "Last updated:
2026-03-30") and ensure any matching footer/header occurrences of the `Last
updated` line are changed to the same date so the document's metadata is
consistent with the PR creation timestamp.

---

Duplicate comments:
In `@README.md`:
- Line 64: The README's Crypto table conflates Web Crypto primitives with the
ECIES implementation; update the Crypto row so it clearly separates Web Crypto
API usage (AES-256-GCM, AES-256-CTR, HKDF-SHA256) from ECIES (implemented via
the eciesjs library, secp256k1). Edit the line that currently reads "Web Crypto
API (AES-256-GCM/CTR, ECIES secp256k1)" to instead list Web Crypto primitives
and explicitly state "ECIES (secp256k1) via eciesjs" so readers can map
implementation to libraries correctly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ff767fc5-7b6e-4c44-8936-c6a814519c4c

📥 Commits

Reviewing files that changed from the base of the PR and between 3bbce98 and 72894f3.

📒 Files selected for processing (10)

• .claude/claude.md
• .planning/DEFERRED.md
• .planning/FEATURES.md
• .planning/ROADMAP.md
• .planning/codebase/CONCERNS.md
• .planning/codebase/TESTING.md
• README.md
• docs/ARCHITECTURE.md
• docs/DATABASE_EVOLUTION_PROTOCOL.md
• docs/FILESYSTEM_SPECIFICATION.md

✅ Files skipped from review due to trivial changes (4)

• docs/DATABASE_EVOLUTION_PROTOCOL.md
• .planning/ROADMAP.md
• docs/FILESYSTEM_SPECIFICATION.md
• .planning/codebase/TESTING.md

🚧 Files skipped from review as they are similar to previous changes (1)

• .planning/FEATURES.md

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@README.md`:
- Line 21: README claims a hard cap of "up to 10 versions" but the metadata
contract in packages/core/src/file/types.ts defines versions?: VersionEntry[]
with no max; either soften the README or enforce the cap. Fix by either (A)
updating README.md to remove or soften the "up to 10 versions" language and
reference the actual metadata shape (versions?: VersionEntry[]) or (B) implement
and document enforcement where versions are created/validated (e.g., add a
maxVersions constant and enforce it in the code paths that append to
VersionEntry arrays such as the file versioning/commit handlers and any
validation functions that construct VersionEntry[]), and update the types or add
runtime validation to reflect the max; ensure you reference versions?:
VersionEntry[] and the module packages/core/src/file/types.ts when making the
change.
- Line 23: The README claim about "30-day soft-delete" is not reflected in code:
inspect packages/core/src/bin/types.ts (the DeletedRecord type / deletedAt
field) and either update README to remove or qualify the 30-day retention
language, or implement an enforcement mechanism (add an expiresAt/ttl field to
the type and a cleanup job that permanently deletes records after 30 days, or
point README to the service that performs TTL cleanup). Make the change
consistent by referencing the DeletedRecord/deletedAt symbol in types.ts and
ensure the README text mentions the exact enforcement point (code path or
background job) or is reworded to avoid implying automatic enforcement.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e3683d5e-753e-40e7-bab5-440465c5c4cc

📥 Commits

Reviewing files that changed from the base of the PR and between 72894f3 and e42e876.

📒 Files selected for processing (3)

• .planning/codebase/CONCERNS.md
• README.md
• docs/FILESYSTEM_SPECIFICATION.md

✅ Files skipped from review due to trivial changes (1)

• docs/FILESYSTEM_SPECIFICATION.md

🚧 Files skipped from review as they are similar to previous changes (1)

• .planning/codebase/CONCERNS.md