Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
f79c65e
docs(19.2): research phase domain
FSM1 Mar 23, 2026
75629ea
docs(phase-19.2): add validation strategy
FSM1 Mar 23, 2026
c82ae23
docs(19.2): create phase plan
FSM1 Mar 23, 2026
4087fa2
docs(19.2-01): capture pre-optimization upload performance baselines
FSM1 Mar 23, 2026
1f9a30f
test(19.2-01): add failing tests for concurrent upload pin orchestration
FSM1 Mar 23, 2026
565dccd
feat(19.2-01): parallelize file IPNS publish and folder metadata update
FSM1 Mar 23, 2026
c197d35
docs(19.2-01): complete baseline capture and concurrent pin orchestra…
FSM1 Mar 23, 2026
1b947a7
perf(19.2-02): configure Kubo with pebbleds datastore for faster writes
FSM1 Mar 23, 2026
11dd124
docs(19.2-02): capture post-optimization baselines with before/after …
FSM1 Mar 23, 2026
f8fff93
docs(19.2-02): complete Kubo pebbleds datastore plan
FSM1 Mar 23, 2026
4e2fe36
docs(19.2): create gap closure plans for PERF-09 registration and sta…
FSM1 Mar 23, 2026
1acabf4
docs(19.2): replan gap closure with three-point local baseline approach
FSM1 Mar 23, 2026
0d482fb
docs(19.2-03): register PERF-09 in REQUIREMENTS.md
FSM1 Mar 23, 2026
aab490b
docs(19.2-03): complete PERF-09 gap closure plan
FSM1 Mar 23, 2026
2b1ecf0
docs: capture todo - Investigate removal of mock-ipns-routing layer
FSM1 Mar 23, 2026
534a8a3
docs(19.2-04): update plan with concurrency probe, inline secrets, so…
FSM1 Mar 23, 2026
67bfc83
docs: capture todo - Load tests should handle 401s with token refresh
FSM1 Mar 23, 2026
5c59248
docs(19.2-04): document three-point local comparison in post-optimiza…
FSM1 Mar 23, 2026
4193fb6
docs(19.2-04): complete three-point local baselines plan
FSM1 Mar 23, 2026
91072c7
docs(19.2): add 75-client pebbleds stress test results
FSM1 Mar 23, 2026
bc23e46
docs(19.2): update verification report after gap closure
FSM1 Mar 23, 2026
8e701d0
refactor(sdk): extract shared test helpers and add ipns:batchPublishF…
FSM1 Mar 23, 2026
44c6bdd
fix(sdk): address PR review comments
FSM1 Mar 23, 2026
8e67807
fix(sdk): fix mock return types and add partial failure test
FSM1 Mar 23, 2026
5066471
fix: address second round of PR review comments
FSM1 Mar 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .planning/REQUIREMENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Requirements for IPFS infrastructure milestone. Each maps to roadmap phases.
- [ ] **PERF-06**: End-to-end user journey timing captured (login-to-vault, upload-to-visible, share-to-accessible)
- [ ] **PERF-07**: k6 load testing scripts simulating concurrent users (upload, download, publish, resolve)
- [ ] **PERF-08**: Capacity thresholds documented with scaling recommendations
- [x] **PERF-09**: Upload operations optimized via concurrent SDK pin orchestration (Promise.allSettled) and Kubo pebbleds datastore, with before/after baselines documented

### SDK Extraction (Phase 19.1)

Expand Down Expand Up @@ -121,6 +122,7 @@ Which phases cover which requirements. Updated during roadmap creation.
| PERF-06 | Phase 22 | Pending |
| PERF-07 | Phase 22 | Pending |
| PERF-08 | Phase 22 | Pending |
| PERF-09 | Phase 19.2 | Complete |
| SDK-01 | Phase 19.1 | Complete |
| SDK-02 | Phase 19.1 | Complete |
| SDK-03 | Phase 19.1 | Complete |
Expand All @@ -135,11 +137,11 @@ Which phases cover which requirements. Updated during roadmap creation.

**Coverage:**

- v1.1 requirements: 36 total
- Mapped to phases: 36
- v1.1 requirements: 37 total
- Mapped to phases: 37
- Unmapped: 0

---

_Requirements defined: 2026-03-07_
_Last updated: 2026-03-20 after Phase 19.1 SDK extraction requirements added_
_Last updated: 2026-03-23 after Phase 19.2 gap closure (PERF-09 registration)_
45 changes: 34 additions & 11 deletions .planning/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Decimal phases appear between their surrounding integers in numeric order.

- [x] **Phase 18: Performance Instrumentation** - Server-side Prometheus histograms and Kubo metrics scraping to establish baselines before any architectural changes (completed 2026-03-07)
- [x] **Phase 19: IPNS Resolution Improvement** - Replace delegated-ipfs.dev with self-hosted Someguy sidecar for reliable IPNS routing, add latency histograms for resolve/publish operations (completed 2026-03-07)
- [x] **Phase 19.2: IPFS Upload Performance Optimization** - Optimize Kubo pinning path (concurrent pins, worker tuning, pin batching) to reduce upload latency (INSERTED) (completed 2026-03-23)
- [ ] **Phase 20: Vault Migration** - Move rootFolderKey to IPFS vault blob v2 format, making the server store zero crypto material
- [ ] **Phase 21: BYO-IPFS Node Support** - User-configurable IPFS pinning endpoint with dual-pin strategy, Settings UI, and connection testing
- [ ] **Phase 22: Performance Baselines Completion** - Client-side timing instrumentation, end-to-end journey timing, k6 load testing, and capacity documentation
Expand Down Expand Up @@ -80,6 +81,27 @@ Plans:
- [ ] 19.1-05-PLAN.md -- Rewire web app hooks and stores to use SDK
- [ ] 19.1-06-PLAN.md -- Remove re-exports, update imports, configure Release Please

### Phase 19.2: IPFS Upload Performance Optimization (INSERTED)

**Goal:** Upload operations are measurably faster by optimizing the Kubo IPFS pinning path — the dominant bottleneck consuming ~95% of upload endpoint latency (~1.73s mean per pin, 3 sequential pins per upload)
**Depends on:** Phase 19.1 (SDK needed for load test benchmarking)
**Requirements**: PERF-09
**Success Criteria** (what must be TRUE):

1. Independent pin operations within a single upload (ciphertext + file metadata) execute concurrently rather than sequentially, reducing per-upload server-side time from ~5.4s to ~3.5s at p50
2. Kubo pin worker configuration is tuned for concurrent load — pin latency under 50 concurrent clients does not exceed 2x the single-client baseline
3. Load test results (same mixed workload scenario, same staging hardware) show measurable throughput improvement over Phase 19 baselines (>15% ops/s increase at 50+ clients)
4. Latency improvements are validated with before/after Prometheus histogram comparisons and documented in baselines

**Plans:** 4/4 plans complete

Plans:

- [x] 19.2-01-PLAN.md -- Capture pre-optimization baselines, parallelize SDK upload pin orchestration with Promise.allSettled
- [x] 19.2-02-PLAN.md -- Switch Kubo datastore to pebbleds, capture post-optimization baselines with before/after comparison
- [x] 19.2-03-PLAN.md -- Register PERF-09 in REQUIREMENTS.md (gap closure)
- [x] 19.2-04-PLAN.md -- Three-point local performance baselines with matched-environment comparison (gap closure)

### Phase 20: Vault Migration

**Goal**: The server stores zero crypto material -- rootFolderKey lives in the IPFS vault blob, making the server a true zero-knowledge relay
Expand Down Expand Up @@ -124,18 +146,19 @@ Plans:
## Progress

**Execution Order:**
Phases execute in numeric order: 18 -> 19 -> 19.1 -> 20 -> 21 -> 22

| Phase | Milestone | Plans Complete | Status | Completed |
| ---------------------------------- | --------- | -------------- | ----------- | ---------- |
| 18. Performance Instrumentation | v1.1 | 2/2 | Complete | 2026-03-07 |
| 19. IPNS Resolution Improvement | v1.1 | 2/2 | Complete | 2026-03-07 |
| 19.1 Extract Core Crypto SDK | v1.1 | 6/6 | Complete | 2026-03-20 |
| 20. Vault Migration | v1.1 | 0/? | Not started | - |
| 21. BYO-IPFS Node Support | v1.1 | 0/? | Not started | - |
| 22. Performance Baselines Complete | v1.1 | 0/? | Not started | - |
Phases execute in numeric order: 18 -> 19 -> 19.1 -> 19.2 -> 20 -> 21 -> 22

| Phase | Milestone | Plans Complete | Status | Completed |
| ----------------------------------------- | --------- | -------------- | ----------- | ---------- |
| 18. Performance Instrumentation | v1.1 | 2/2 | Complete | 2026-03-07 |
| 19. IPNS Resolution Improvement | v1.1 | 2/2 | Complete | 2026-03-07 |
| 19.1 Extract Core Crypto SDK | v1.1 | 6/6 | Complete | 2026-03-20 |
| 19.2 IPFS Upload Performance Optimization | v1.1 | 4/4 | Complete | 2026-03-23 |
| 20. Vault Migration | v1.1 | 0/? | Not started | - |
| 21. BYO-IPFS Node Support | v1.1 | 0/? | Not started | - |
| 22. Performance Baselines Complete | v1.1 | 0/? | Not started | - |

---

_Roadmap created: 2026-03-07_
_Last updated: 2026-03-20_
_Last updated: 2026-03-23_
29 changes: 18 additions & 11 deletions .planning/STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
gsd_state_version: 1.0
milestone: v1.1
milestone_name: milestone
status: completed
last_updated: '2026-03-20T06:06:34.905Z'
status: unknown
last_updated: '2026-03-23T22:18:25.552Z'
progress:
total_phases: 6
completed_phases: 6
total_plans: 10
completed_plans: 10
total_phases: 7
completed_phases: 4
total_plans: 14
completed_plans: 14
---

# Project State
Expand All @@ -18,18 +18,18 @@ progress:
See: .planning/PROJECT.md (updated 2026-03-07)

**Core value:** Zero-knowledge privacy -- files encrypted client-side, server never sees plaintext
**Current focus:** Phase 19.1extract-core-crypto-sdk-as-shared-package
**Current focus:** Phase 19.2ipfs-upload-performance-optimization

## Current Position

Phase: 19.1 (extract-core-crypto-sdk-as-shared-package) — COMPLETE
Plan: 6 of 6
Phase: 19.2 (ipfs-upload-performance-optimization) — COMPLETE
Plan: 4 of 4

## Performance Metrics

**Velocity:**

- Total plans completed: 157 (72 M1 + 83 M2 + 2 M3)
- Total plans completed: 160 (72 M1 + 83 M2 + 5 M3)
- Average duration: 5.5 min
- Total execution time: ~16.5 hours

Expand All @@ -42,6 +42,10 @@ Plan: 6 of 6
| 19.1 | 03 | 12min | 3 | 18 |
| 19.1 | 04 | 10min | 2 | 14 |
| 19.1 | 06 | 13min | 3 | 52 |
| 19.2 | 01 | 6min | 2 | 4 |
| 19.2 | 02 | 12min | 3 | 3 |
| 19.2 | 03 | 1min | 1 | 1 |
| 19.2 | 04 | 71min | 2 | 1 |

## Accumulated Context

Expand All @@ -66,10 +70,13 @@ Recent for v1.1:
- Bin/share operations take explicit context objects (BinOperationContext, ShareOperationContext) instead of Zustand stores
- Share module accepts callback functions for API calls to stay transport-decoupled
- Moved @cipherbox/core from dependencies to devDependencies in crypto (test-only cross-package assertions)
- Kubo pebbleds datastore (LSM-tree) configured via IPFS_PROFILE=server,pebbleds; requires fresh volume on deploy
- SDK concurrent pins require pebbleds datastore (synergistic); concurrent pins alone cause regression at 50 clients

### Roadmap Evolution

- Phase 19.1 inserted after Phase 19: Extract core crypto SDK as shared package (URGENT)
- Phase 19.2 inserted after Phase 19: IPFS Upload Performance Optimization (URGENT) — concurrent pins, Kubo worker tuning, pin batching to address ~95% bottleneck in upload path identified by Phase 19 baselines

### Open Concerns

Expand All @@ -89,4 +96,4 @@ All M2 blockers resolved. See `.planning/milestones/m2/m2-v1.0-production-MILEST

---

Last updated: 2026-03-20 after completing 19.1-06 (final cleanup: removed re-exports, Release Please, Codecov, READMEs)
Last updated: 2026-03-23 after completing 19.2-04 (three-point local performance baselines — Phase 19.2 complete)
Loading
Loading