feat: phase 22 — performance baselines completion#355
Merged
Conversation
Entire-Checkpoint: 76b95ac159b3
Entire-Checkpoint: 35000a89c030
Entire-Checkpoint: e4a7fc6a9315
Entire-Checkpoint: d83247189e1d
Entire-Checkpoint: 3f5c191108bb
- 6 test cases covering withPerf, markStart, markEnd - Tests verify env gating, mark cleanup, error propagation - All tests fail (RED phase) — perf.ts not yet created Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 33c88d096f16
… wrappers - PERF_ENABLED constant evaluated at module load (no-op in production) - markStart/markEnd create and clean up performance marks - withPerf async wrapper for transparent instrumentation - __CIPHERBOX_PERF__ global opt-in for production debugging - All 6 unit tests pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: ba944ef18fb0
- 5 test cases covering pass/fail, p95 violations, error rate violations - Tests skip operations not in metrics (no false positives) - Violation messages include operation name, observed, and threshold values Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 197eddfb5b35
- Login-to-vault: mock wallet auth through vault load with phase breakdown - Upload-to-visible: 100KB file upload through file list appearance - Share-to-accessible: share creation through recipient access with phase breakdown - All timings use performance.now() with JOURNEY_TIMING: JSON output prefix - Graceful failure handling for multi-account share journey Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: ef87d5191af2
- ThresholdConfig type with operation, p95MaxMs, errorRateMax fields - checkThresholds function compares metrics against limits - Skips operations not in metrics, returns descriptive violations - All 5 unit tests pass Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 35feaa95419d
- Login-to-vault, upload-to-visible, share-to-accessible sections - Capture information table with environment details - PENDING markers for values to be filled after test execution - How to Capture section with playwright command and grep pattern - Historical comparison table for SDK vs E2E overhead analysis Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 946420b9dd05
- upload-throughput: uploadFile p95 <= 10s, error rate <= 5% - mixed-workload: uploadFile p95 <= 10s, createFolder p95 <= 5s, error <= 10% - ipns-publish-storm: createFolder p95 <= 10s, error <= 10% - sustained-load: uploadFile p95 <= 10s, createFolder p95 <= 5s, error <= 5% - spike-test: burst phase uploadFile/createFolder p95 <= 15s, error <= 15% - Thresholds are 2-3x observed Phase 19.2 baselines to avoid CI flakiness Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 4c58196e9d12
…appers - upload:full, download:full (upload/download modules) - ipfs:upload, ipfs:download (IPFS module) - ipns:publish, ipns:batch-publish, ipns:resolve (IPNS module) - folder:fetch-decrypt, folder:load, folder:update-publish (folder module) - All function signatures and return values unchanged - All 34 existing tests pass, build succeeds Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: cfaa84c37d6f
- SUMMARY.md with self-check passed - STATE.md updated with plan progress and metrics - ROADMAP.md updated with phase 22 progress - REQUIREMENTS.md marks PERF-06 complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 4a516da82d28
- Observed limits from Phases 18, 19, 19.2 with real baseline data - Infrastructure bottleneck analysis (Kubo pin ~95% of upload time) - Scaling recommendations with trigger thresholds and actions - Growth projections with formulas for storage, IPNS names, DB rows - Cost estimates for 100/1,000/10,000 user deployments - Load test threshold reference table linked to thresholds.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: c54dcb4a1bc8
- SUMMARY.md with task commits and self-check - STATE.md updated with metrics and decisions - ROADMAP.md progress updated - PERF-05 requirement marked complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: a332b0bbb7b0
- 22-03-SUMMARY.md with task commits and self-check - STATE.md updated: plan 3/3, progress 100%, metrics, decisions - ROADMAP.md updated: phase 22 complete (3/3 plans) - REQUIREMENTS.md: PERF-07 and PERF-08 marked complete Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 8facff4bded9
Entire-Checkpoint: d56174c486b8
Entire-Checkpoint: 083026da5d55
- Fix PerformanceMeasure leak: clear measures after creation in perf.ts - Extract expectThresholdsPassed() helper, replace 5 identical 8-line blocks - Instrument unpinFromIpfs with withPerf for consistency - Remove unused partial field and tighten account scope in journey-timing - Remove unused imports (expect, PrivateKeyAccount) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: f8f1237175c6
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (17)
✅ Files skipped from review due to trivial changes (13)
🚧 Files skipped from review as they are similar to previous changes (4)
WalkthroughPhase 22 (Performance Baselines Completion) is marked complete. The PR adds SDK client-side performance instrumentation ( Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #355 +/- ##
===========================================
+ Coverage 56.55% 70.21% +13.66%
===========================================
Files 115 104 -11
Lines 8342 6138 -2204
Branches 748 892 +144
===========================================
- Hits 4718 4310 -408
+ Misses 3445 1624 -1821
- Partials 179 204 +25
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Completes Phase 22 performance baselines work by adding SDK-level Performance API instrumentation, Playwright E2E journey timing output, load-test threshold assertions, and capacity planning documentation.
Changes:
- Added
withPerf/markStart/markEndutilities and wrapped keysdk-coreoperations (upload/download/IPFS/IPNS/folder) for timing visibility. - Introduced Playwright “journey timing” spec that outputs structured
JOURNEY_TIMING:JSON lines for baseline capture. - Added load-test threshold checking utilities + unit tests, integrated thresholds into all load scenarios, and documented capacity baselines in
docs/CAPACITY.md.
Reviewed changes
Copilot reviewed 31 out of 31 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/web-e2e/tests/journey-timing.spec.ts | Adds 3 timed E2E journeys with structured JSON output. |
| tests/load/src/scenarios/upload-throughput.test.ts | Runs aggregate report and asserts p95/error-rate thresholds. |
| tests/load/src/scenarios/sustained-load.test.ts | Adds scenario-specific threshold assertions. |
| tests/load/src/scenarios/spike-test.test.ts | Adds burst-phase threshold assertions for spike scenario. |
| tests/load/src/scenarios/mixed-workload.test.ts | Adds threshold assertions for mixed workload operations. |
| tests/load/src/scenarios/ipns-publish-storm.test.ts | Adds threshold assertions for IPNS contention scenario. |
| tests/load/src/harness/thresholds.ts | Implements threshold evaluation + failure helper. |
| tests/load/src/harness/thresholds.test.ts | Unit tests for threshold evaluation behavior/messages. |
| packages/sdk-core/src/perf.ts | Adds Performance API wrapper utilities and gating. |
| packages/sdk-core/src/upload/index.ts | Wrapes uploadFile with perf measurement. |
| packages/sdk-core/src/download/index.ts | Wrapes downloadAndDecrypt with perf measurement. |
| packages/sdk-core/src/ipfs/index.ts | Wraps IPFS upload/download/unpin with perf measurement. |
| packages/sdk-core/src/ipns/index.ts | Wraps IPNS publish/batch/resolve with perf measurement. |
| packages/sdk-core/src/folder/index.ts | Wraps folder load/fetch+decrypt/update+publish with perf measurement. |
| packages/sdk-core/src/tests/perf.test.ts | Adds unit tests validating perf gating/cleanup/error propagation. |
| docs/CAPACITY.md | Adds capacity model doc consolidating observed limits and scaling guidance. |
| .planning/phases/22-performance-baselines-completion/22-VERIFICATION.md | Adds verification report for Phase 22 deliverables. |
| .planning/phases/22-performance-baselines-completion/22-VALIDATION.md | Adds Phase 22 validation strategy/spec. |
| .planning/phases/22-performance-baselines-completion/22-RESEARCH.md | Adds research notes and rationale for approach. |
| .planning/phases/22-performance-baselines-completion/22-CONTEXT.md | Adds phase context and constraints. |
| .planning/phases/22-performance-baselines-completion/22-03-SUMMARY.md | Adds plan 03 summary (thresholds + capacity). |
| .planning/phases/22-performance-baselines-completion/22-03-PLAN.md | Adds plan 03 execution doc. |
| .planning/phases/22-performance-baselines-completion/22-02-SUMMARY.md | Adds plan 02 summary (journey timing + baselines template). |
| .planning/phases/22-performance-baselines-completion/22-02-PLAN.md | Adds plan 02 execution doc. |
| .planning/phases/22-performance-baselines-completion/22-01-SUMMARY.md | Adds plan 01 summary (SDK perf instrumentation). |
| .planning/phases/22-performance-baselines-completion/22-01-PLAN.md | Adds plan 01 execution doc. |
| .planning/baselines/22-journey-baselines.md | Adds template baseline doc with [PENDING] placeholders. |
| .planning/STATE.md | Updates current phase/plan state tracking. |
| .planning/ROADMAP.md | Marks Phase 22 complete and adds plan list. |
| .planning/REQUIREMENTS.md | Marks PERF-05..08 complete. |
| .planning/PROJECT.md | Updates “Last updated” to reflect Phase 22 completion. |
Captured against staging (api-staging.cipherbox.cc) on 2026-03-25: - login-to-vault: 23.6s (23.5s wallet auth + 86ms vault load) - upload-to-visible: 1.4s (100KB file) - share-to-accessible: 3.0s (2.2s share create + 0.8s recipient access) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: a69ecb4e0a1a
- Make perf mark names unique per invocation (concurrent-safe) - Tighten PERF_ENABLED to require defined NODE_ENV string (safe in browser prod) - Fix 404 handling to check error.response.status for AxiosError - Fix ROADMAP plan checkboxes, table alignment, and STATE status - Update PERF-07 requirement text from k6 to Vitest harness Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: e5c4e67212f2
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
tests/load/src/scenarios/spike-test.test.ts (1)
15-15: Threshold integration looks correct, butuploadFilethreshold is unreachable.The spike test uses
runFolderWorkload, which only recordscreateFolder,renameItem, anddeleteItemoperations (pertests/load/src/workloads/folder-workload.ts). TheuploadFilethreshold at line 82 will never be checked sincecheckThresholdssilently skips operations not present in the metrics.Consider removing the
uploadFileentry to avoid confusion:🔧 Suggested fix
const THRESHOLDS: ThresholdConfig[] = [ - { operation: 'uploadFile', p95MaxMs: 15_000, errorRateMax: 0.15 }, { operation: 'createFolder', p95MaxMs: 15_000, errorRateMax: 0.15 }, ];Also applies to: 79-86
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/load/src/scenarios/spike-test.test.ts` at line 15, The spike test declares an unreachable "uploadFile" threshold because the scenario uses runFolderWorkload which only emits createFolder, renameItem, and deleteItem metrics; remove the "uploadFile" entry from the thresholds in spike-test.test.ts (or replace it with one of the emitted ops) so expectThresholdsPassed/checkThresholds won't silently skip it, and keep references to runFolderWorkload and the uploadFile threshold key when editing so reviewers can locate the change.tests/load/src/harness/thresholds.test.ts (1)
24-135: Add explicit coverage forcount = 0error-rate handling.The implementation has a divide-by-zero guard path; add a dedicated test to lock that behavior (error rate should remain
0and not violate unless latency does).As per coding guidelines, "**/*.test.ts: Focus on test coverage, edge cases, and test quality."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/load/src/harness/thresholds.test.ts` around lines 24 - 135, Add a test case in thresholds.test.ts that covers the count = 0 path for checkThresholds: create an OperationMetrics via makeMetrics with operation 'uploadFile', count: 0 and errors: 0 (or errors non-zero to still ensure guard path yields 0% if designed that way), supply a ThresholdConfig for that operation with a low errorRateMax to ensure no divide-by-zero causes a false violation, call checkThresholds(metrics, thresholds) and assert result.passed is true and that no 'error rate' violation appears; reference checkThresholds, makeMetrics, and ThresholdConfig when adding the test.packages/sdk-core/src/__tests__/perf.test.ts (1)
70-100: Add a test for production opt-in (__CIPHERBOX_PERF__ = true).You cover “enabled in test” and “disabled in production,” but not the explicit production opt-in path. Add one assertion for this branch to prevent regression in PERF gating behavior.
As per coding guidelines, "**/*.test.ts: Focus on test coverage, edge cases, and test quality."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/sdk-core/src/__tests__/perf.test.ts` around lines 70 - 100, Add a test case that covers the production opt-in path by setting process.env.NODE_ENV = 'production' and setting globalThis.__CIPHERBOX_PERF__ = true, then import the perf helpers and assert behavior when enabled: for withPerf('prod:optin', ...) assert it returns the wrapped value and that performance.getEntriesByType('measure') contains entries whose names start with 'cipherbox:'; and for markStart('optin:test') assert a non-empty start mark and that markEnd('optin:test', start) returns a non-null measure. Target the existing helpers withPerf, markStart, and markEnd in your new test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.planning/ROADMAP.md:
- Line 27: Update the Phase 22 description text that currently reads “k6 load
testing” to “Vitest-based SDK load harness” and refine the PERF-07 entry to
state “Vitest-based SDK load harness” (preserving the same concurrent actions
scope: upload/download/publish/resolve); locate the string "Phase 22:
Performance Baselines Completion" and the PERF-07 mention in
.planning/ROADMAP.md and replace the k6 wording with the new phrasing while
leaving all scope and checklist semantics unchanged.
- Line 160: The roadmap entry is stale: update the checklist item that currently
reads "instrument 10 sdk-core functions" to reflect the new total of 11
functions because the PR also instruments unpinFromIpfs; specifically edit the
ROADMAP.md entry referencing 22-01-PLAN.md / perf.ts to say "instrument 11
sdk-core functions" (or otherwise note that unpinFromIpfs was added) so the
document matches the implemented instrumentation.
In `@packages/sdk-core/src/__tests__/perf.test.ts`:
- Around line 9-20: Remove the repeated `(globalThis as any).__CIPHERBOX_PERF__`
casts by declaring the property on the global type for the test module and then
use the typed global directly; add a module-level `declare global { var
__CIPHERBOX_PERF__: boolean | undefined }` (or appropriate type) and replace all
uses (e.g., `originalPerfFlag = (globalThis as any).__CIPHERBOX_PERF__`, the
afterEach restore block, and the other occurrences around lines 73 and 91) with
`globalThis.__CIPHERBOX_PERF__` so ESLint warnings go away while preserving the
existing logic that reads, deletes, or assigns the flag (refer to the
`originalPerfFlag` variable and the afterEach restore logic).
---
Nitpick comments:
In `@packages/sdk-core/src/__tests__/perf.test.ts`:
- Around line 70-100: Add a test case that covers the production opt-in path by
setting process.env.NODE_ENV = 'production' and setting
globalThis.__CIPHERBOX_PERF__ = true, then import the perf helpers and assert
behavior when enabled: for withPerf('prod:optin', ...) assert it returns the
wrapped value and that performance.getEntriesByType('measure') contains entries
whose names start with 'cipherbox:'; and for markStart('optin:test') assert a
non-empty start mark and that markEnd('optin:test', start) returns a non-null
measure. Target the existing helpers withPerf, markStart, and markEnd in your
new test.
In `@tests/load/src/harness/thresholds.test.ts`:
- Around line 24-135: Add a test case in thresholds.test.ts that covers the
count = 0 path for checkThresholds: create an OperationMetrics via makeMetrics
with operation 'uploadFile', count: 0 and errors: 0 (or errors non-zero to still
ensure guard path yields 0% if designed that way), supply a ThresholdConfig for
that operation with a low errorRateMax to ensure no divide-by-zero causes a
false violation, call checkThresholds(metrics, thresholds) and assert
result.passed is true and that no 'error rate' violation appears; reference
checkThresholds, makeMetrics, and ThresholdConfig when adding the test.
In `@tests/load/src/scenarios/spike-test.test.ts`:
- Line 15: The spike test declares an unreachable "uploadFile" threshold because
the scenario uses runFolderWorkload which only emits createFolder, renameItem,
and deleteItem metrics; remove the "uploadFile" entry from the thresholds in
spike-test.test.ts (or replace it with one of the emitted ops) so
expectThresholdsPassed/checkThresholds won't silently skip it, and keep
references to runFolderWorkload and the uploadFile threshold key when editing so
reviewers can locate the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: ebc8acc9-9e25-4442-aa9f-959e326bf10c
📒 Files selected for processing (31)
.planning/PROJECT.md.planning/REQUIREMENTS.md.planning/ROADMAP.md.planning/STATE.md.planning/baselines/22-journey-baselines.md.planning/phases/22-performance-baselines-completion/22-01-PLAN.md.planning/phases/22-performance-baselines-completion/22-01-SUMMARY.md.planning/phases/22-performance-baselines-completion/22-02-PLAN.md.planning/phases/22-performance-baselines-completion/22-02-SUMMARY.md.planning/phases/22-performance-baselines-completion/22-03-PLAN.md.planning/phases/22-performance-baselines-completion/22-03-SUMMARY.md.planning/phases/22-performance-baselines-completion/22-CONTEXT.md.planning/phases/22-performance-baselines-completion/22-RESEARCH.md.planning/phases/22-performance-baselines-completion/22-VALIDATION.md.planning/phases/22-performance-baselines-completion/22-VERIFICATION.mddocs/CAPACITY.mdpackages/sdk-core/src/__tests__/perf.test.tspackages/sdk-core/src/download/index.tspackages/sdk-core/src/folder/index.tspackages/sdk-core/src/ipfs/index.tspackages/sdk-core/src/ipns/index.tspackages/sdk-core/src/perf.tspackages/sdk-core/src/upload/index.tstests/load/src/harness/thresholds.test.tstests/load/src/harness/thresholds.tstests/load/src/scenarios/ipns-publish-storm.test.tstests/load/src/scenarios/mixed-workload.test.tstests/load/src/scenarios/spike-test.test.tstests/load/src/scenarios/sustained-load.test.tstests/load/src/scenarios/upload-throughput.test.tstests/web-e2e/tests/journey-timing.spec.ts
- Update ROADMAP Phase 22 description from k6 to Vitest-based load testing - Update instrumented function count from 10 to 11 (unpinFromIpfs added) - Replace `as any` casts on globalThis with `declare global` type declaration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 60737b3c4487
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 10337a3e9326
Owner
Author
|
@coderabbitai please review |
|
🧠 Learnings used✅ Actions performedReview triggered.
|
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: f0cb8c856004
Resolves conflicts between Phase 21 BYO-IPFS completion and Phase 22 performance baselines. Both phases now marked complete in ROADMAP, STATE, and PROJECT. Upload pinFn override from Phase 21 preserved alongside withPerf instrumentation from Phase 22. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 18d127586f12
Allows distinguishing CipherBox-pinned uploads (ipfs:upload) from BYO-pinned uploads (ipfs:upload:byo) in the Performance timeline, enabling diagnosis of BYO-specific latency issues. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: ef23b97d131d
Consistent naming with ipfs:upload:byo for clear distinction in Performance timeline. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Entire-Checkpoint: 3b4cc5bc6dba
| - pinWithMode treats Pinata like Kubo: direct upload bypasses CipherBox relay entirely | ||
| - Connection test probe order updated: Kubo -> Pinata -> PSA; pinata.cloud URLs skip Kubo probe | ||
| - BYO Pinata baselines: pin p50=2.0s (+47% vs local Kubo), tail latency p99 13.5% better, 98% CipherBox API load reduction per file | ||
| - perf.ts PERF_ENABLED evaluated once at module load (zero overhead in production); **CIPHERBOX_PERF** global for opt-in production debugging |
There was a problem hiding this comment.
The state summary references a CIPHERBOX_PERF global flag, but the actual opt-in flag used by sdk-core perf gating is globalThis.__CIPHERBOX_PERF__ (double-underscore). This mismatch will mislead anyone trying to enable perf marks in production/debug builds; please update the flag name here to match the implementation.
Suggested change
| - perf.ts PERF_ENABLED evaluated once at module load (zero overhead in production); **CIPHERBOX_PERF** global for opt-in production debugging | |
| - perf.ts PERF_ENABLED evaluated once at module load (zero overhead in production); `globalThis.__CIPHERBOX_PERF__` global for opt-in production debugging |
| - Connection test probe order updated: Kubo -> Pinata -> PSA; pinata.cloud URLs skip Kubo probe | ||
| - BYO Pinata baselines: pin p50=2.0s (+47% vs local Kubo), tail latency p99 13.5% better, 98% CipherBox API load reduction per file | ||
| - perf.ts PERF_ENABLED evaluated once at module load (zero overhead in production); **CIPHERBOX_PERF** global for opt-in production debugging | ||
| - Load test thresholds set at 2-3x observed baselines; spike test most generous (15s/15%); vitest expect() for CI failure on breach |
There was a problem hiding this comment.
This line says load-test thresholds use vitest expect() to fail CI on breach, but the scenarios now call expectThresholdsPassed(...), which logs violations and throws an Error (no vitest expect() involved). Please update the text to reflect the actual enforcement mechanism so the state doc stays accurate.
Suggested change
| - Load test thresholds set at 2-3x observed baselines; spike test most generous (15s/15%); vitest expect() for CI failure on breach | |
| - Load test thresholds set at 2-3x observed baselines; spike test most generous (15s/15%); enforced via expectThresholdsPassed(...) which logs violations and throws on breach to fail CI |
| expected: 'JOURNEY_TIMING: JSON lines printed for login-to-vault, upload-to-visible, share-to-accessible, and summary; each timing under its sanity-check limit' | ||
| why_human: 'Requires running API + frontend locally; Playwright tests cannot be verified without live services' | ||
| - test: 'Open browser DevTools Performance tab while using the app and start a recording' | ||
| expected: 'cipherbox:upload:full, cipherbox:ipfs:upload, cipherbox:folder:update-publish etc. appear as named measures in the timeline' |
There was a problem hiding this comment.
The manual verification expectation lists cipherbox:ipfs:upload, but the SDK instrumentation uses more specific operation names (e.g. ipfs:upload:cipherbox and ipfs:upload:byo, which become cipherbox:ipfs:upload:cipherbox / cipherbox:ipfs:upload:byo). Please update this example list so someone following the steps in DevTools knows what to look for.
Suggested change
| expected: 'cipherbox:upload:full, cipherbox:ipfs:upload, cipherbox:folder:update-publish etc. appear as named measures in the timeline' | |
| expected: 'cipherbox:upload:full, cipherbox:ipfs:upload:cipherbox, cipherbox:ipfs:upload:byo, cipherbox:folder:update-publish etc. appear as named measures in the timeline' |
| | 4 | Existing sdk-core function signatures and return values are unchanged | VERIFIED | `withPerf` is a transparent wrapper using `return withPerf(op, async () => { ...existing body... })` - signatures identical | | ||
| | 5 | Login-to-vault journey timing is captured as wall-clock milliseconds in a Playwright test | VERIFIED | `journey-timing.spec.ts` Journey 1 uses `performance.now()`, outputs `JOURNEY_TIMING:` JSON with `walletAuthMs` and `vaultLoadMs` | | ||
| | 6 | Upload-to-visible and share-to-accessible journey timings are captured | VERIFIED | Journey 2 and Journey 3 both present with `performance.now()` timing and structured JSON output | | ||
| | 7 | Load test scenarios have automated pass/fail thresholds that fail the test when p95 latency or error rate exceeds defined limits | VERIFIED | All 5 scenarios import `checkThresholds` and call `expect(thresholdResult.passed).toBe(true)` with violation messages | |
There was a problem hiding this comment.
In the "Observable Truths" table, the evidence for threshold enforcement says scenarios import checkThresholds and call expect(thresholdResult.passed).toBe(true), but the current implementation imports expectThresholdsPassed and throws on violations. Please update the evidence text to match the current code so the verification report remains trustworthy.
Suggested change
| | 7 | Load test scenarios have automated pass/fail thresholds that fail the test when p95 latency or error rate exceeds defined limits | VERIFIED | All 5 scenarios import `checkThresholds` and call `expect(thresholdResult.passed).toBe(true)` with violation messages | | |
| | 7 | Load test scenarios have automated pass/fail thresholds that fail the test when p95 latency or error rate exceeds defined limits | VERIFIED | All 5 scenarios import `expectThresholdsPassed` and rely on it to throw when any threshold is violated; thrown errors carry violation messages | |
7 tasks
Merged
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
withPerfwrapper using browser Performance API (environment-gated viaPERF_ENABLED), instrumented 11 sdk-core functions across upload, download, IPFS, IPNS, and folder modulescheckThresholdsmodule withexpectThresholdsPassedhelper integrated into all 5 load test scenarios, plus 347-line capacity model documentRequirement Coverage
packages/sdk-core/src/perf.ts+ 5 instrumented modulestests/web-e2e/tests/journey-timing.spec.tstests/load/src/harness/thresholds.ts+ 5 scenariosdocs/CAPACITY.mdTest plan
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Tests
Documentation