diff --git a/docs/codebase-index.md b/docs/codebase-index.md index c7d597ed5a..7e4d656d97 100644 --- a/docs/codebase-index.md +++ b/docs/codebase-index.md @@ -336,6 +336,7 @@ One shared composer (`master-search-header.tsx`) serves every mode. Placement: | Search/RAG roadmap | `docs/search-rag-master-plan.md` | | Reindex operations | `docs/reindex-runbook.md` | | Production readiness | `docs/production-readiness-checklist.md` | +| Universal task ledger | `docs/outstanding-issues.md` | | Capacity / scale-up | `docs/capacity-review.md`, `docs/auth-connection-cap-runbook.md` | | Frontend architecture | `docs/frontend-architecture.md` | | Repo audit (2026-07-01) | `docs/audit/repo-audit-2026-07-01.md` | diff --git a/docs/outstanding-issues.md b/docs/outstanding-issues.md index c4b899e310..e88a2248e5 100644 --- a/docs/outstanding-issues.md +++ b/docs/outstanding-issues.md @@ -11,6 +11,9 @@ Durable, cross-session memory of everything still outstanding for this repo: ope - Say `/issues` in Claude Code → the skill reads this file and states the open items back, grouped by priority with a one-line summary count. Nothing is mutated on a plain read. +- Use the **Recommended execution queue** below for current order, acuity, intelligence, timing, + estimates, gates, success criteria, verification, and stopping conditions. This file is the single + universal ledger; do not create a parallel task-ledger document. - `/issues add …`, `/issues done `, `/issues capture`, and friends mutate the tables below. The full command surface lives in the skill file. - Every mutation keeps this file committed so the memory survives across sessions and worktrees. @@ -27,7 +30,66 @@ Durable, cross-session memory of everything still outstanding for this repo: ope - Resolving an item moves its row to **Resolved / archive** with the date and a one-line outcome — rows are archived, not deleted, so the history stays auditable. - + + +## Recommended execution queue + +This queue contains only work still recommended after reconciliation against locally cached `main` +at `6a56a89ae2df14b1dd3c705dc65848b50961f30b` on 2026-07-24. Provider state was not refreshed. +Re-check the source row and current `main` before starting. Lower available order numbers come first; +heavyweight verification remains serialized. + +- **P1:** close before real-patient clinical use or the next release decision. +- **P2:** material correctness, clinical-quality, accessibility, or ingestion reliability. +- **P3:** conditional work; act only when its trigger or owner decision exists. +- **Sol/xhigh:** clinical, privacy, RAG-safety, or ambiguous cross-system reasoning. +- **Sol/high:** database, ingestion, release, or test-contract implementation. +- **Terra/high:** bounded frontend, documentation, product, or operational implementation. +- Estimates are focused hands-on time and exclude approval, provider, CI, and review waits. + +There is no confirmed P0. `#052` and `#053` are the two P1 items. + +| Order | Source | Recommended outcome | Acuity | Classification/state | Intelligence | When | Hands-on estimate | +| ----: | ------------------------- | ----------------------------------------------------------------------- | ------ | ------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------- | --------------------------------------------------- | +| 1 | #052 | Align the Safety Plan Generator with the no-patient-data contract | P1 | Required now | Sol/xhigh + privacy/clinical review | Now; before real-patient use | 3–5 hours | +| 2 | #053 | Close or explicitly defer the APP 8/DPA/ZDR governance basis | P1 | Requires user/operator/provider decision | Sol/xhigh + counsel + account owner | Start now; complete before real-patient use | 2–4 hours preparation; external elapsed time varies | +| 3 | #054 | Fail closed when answer relevance metadata is absent | P2 | Required now | Sol/xhigh | Next local clinical-safety change | 2–4 hours | +| 4 | #030 | Require distinct documents for distinct comparison slots | P2 | Required now | Sol/high | Before trusting another admission/discharge verdict | 2–4 hours | +| 5 | #055 | Recover `queued` documents that have no open ingestion job | P2 | Required now | Sol/high | Before claiming at-least-once ingestion | 1–2 days | +| 6 | #026 | Activate the merged update-only document-change trigger safely | P2 | Requires provider approval | Sol/high + DB operator | After #055's contract is settled | 2–4 hours plus provider setup | +| 7 | #056 | Reconcile and verify the preserved browser/contrast patch | P2 | Recommended; preserved | Terra/high | After higher-acuity local fixes; before the release UI gate | 4–8 hours | +| 8 | #058 | Complete the compact document source-text accordion handoff | P2 | Defer until the user explicitly resumes it | Terra/high | On explicit return to the paused document-viewer task | 0.5–1.5 days | +| 9 | #051, #023 | Compare the scheduled structured canary with run `30018289898` | P2 | Defer until scheduled artifact; provider approval | Sol/xhigh + RAG reviewer | After 2026-07-26 18:00 UTC | 1–2 hours | +| 10 | #018, #029 | Reproduce and address each remaining RAG residual separately | P2 | Defer until #051/#023 | Sol/xhigh + clinical/RAG review | Only after the comparable canary review | 1–2 days per proven mechanism | +| 11 | #022 | Decide BMJ attestation policy and refresh high-impact source governance | P2 | Requires clinical/operator decision | Sol/xhigh + clinical owner | After an approved current-state read | 1–3 days; review elapsed time varies | +| 12 | #033 | Reconsider governance metadata in the model prompt | P3 | Defer until #022 and a harm reproducer | Sol/xhigh + clinical review | Only after metadata coverage improves | 1–2 days | +| 13 | #057 | Run an exact-SHA release gate, including runtime hash-secret proof | P2 | Requires provider approval | Sol/high + release operator | After release-candidate fixes land | 2–4 hours; hosted elapsed time varies | +| 14 | #011 | Change Supabase Auth allocation before compute scale-up | P3 | Defer until approved scale-up | Sol/high + Supabase operator | Immediately before the first compute resize | 1–2 hours plus soak | +| 15 | #007 | Choose the canonical Tools entry point | P3 | Requires user/product decision | Terra/high + product owner | When navigation ownership is available | 30–60 minutes decision; 2–4 hours implementation | +| 16 | #009 | Decide whether `/api/jobs` remains an ops-only surface | P3 | Requires operator decision | Terra/high + operations owner | Before expanding or documenting job operations | 30–60 minutes; 2–4 hours if removal/docs follow | +| 17 | #025 | Activate only approved webhooks with named owners | P3 | Requires provider/operator decision | Sol/high + DevOps owner | After #026 and secret/alert ownership decisions | 2–4 hours plus provider setup | +| 18 | #037 | Decide the D5 all-claims trust cap | P3 | Requires clinical/product decision | Sol/xhigh + clinical/product owners | Only if owners want a universal trust-policy change | 1–2 hours decision; 0.5–1 day implementation | +| 19 | #010 | Build one specific “Coming soon” feature | P3 | Optional/product-triggered | Terra/high | Only after an approved feature brief | 0.5–3 days per feature | +| 20 | #017 | Measure production Web Vitals | P3 | Optional/provider approval | Terra/high + performance reviewer | Only when performance prioritisation is requested | 2–4 hours | +| 21 | #012, #013 | Reduce only payloads proven material by #017 | P3 | Defer until measurement | Sol/high | Only after a user-visible bottleneck is measured | 0.5–3 days per route | +| 22 | #027 | Add an independently owned uptime monitor | P3 | Optional/provider decision | Terra/high + operations owner | When alert ownership and cost are approved | 1–2 hours | +| 23 | #038, #039 | Consolidate shared comparison or toolbar behavior | P3 | Optional/triggered | Terra/high + design owner | Before adding another genuinely duplicate surface | 1–3 days per contract | +| 24 | #040 | Establish a small visual-regression baseline set | P3 | Optional/team decision | Terra/high + UI owner | After baseline ownership/update policy are accepted | 1–2 days | +| 25 | Operator backlog/runbooks | Perform staging soak or DR reconstitution only for a real rehearsal | P3 | Defer until release/rehearsal trigger | Sol/high + operator | When a real candidate or scheduled rehearsal exists | 0.5–2 days | + +### Execution contract + +- Every retained row's **Detail / next action** below defines its smallest scope. The linked source or + runbook supplies deeper implementation steps. +- Success means the stated outcome is observable, focused verification passes, and required human or + provider evidence is recorded without secrets. +- Local changes use focused tests first, then `verify:cheap` when warranted. UI changes additionally + use identity-verified `ensure` and the appropriate `verify:ui` slice. RAG, ingestion, privacy, + production-environment, and clinical-output changes also run production-readiness. +- Stop when a dependency, approval, reproducer, identity check, migration replay, or safety gate is + absent. Do not broaden scope, spend on live checks, or contact providers to force a task closed. +- On completion, move the source row to **Resolved / archive** and remove it from this queue. New + reproducible evidence may create a new narrower issue; historical claims do not automatically reopen. ## Open items @@ -37,6 +99,13 @@ Durable, cross-session memory of everything still outstanding for this repo: ope | ID | Pri | Type | Summary | Detail / next action | Source | Added | | ---- | --- | ----- | --------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| #052 | P1 | issue | Safety Plan Generator contradicts the privacy contract | **Outcome:** the tool, privacy notice, PIA, and tests agree on whether patient identifiers may be entered, copied, printed, or saved. `patient-safety-plan.tsx` asks for “Patient (name or initials)” and produces a patient copy, while `/privacy` and the PIA say the product does not ask for patient data. **Next:** obtain a privacy/clinical decision, then default to identifier-free behavior unless transient identifier processing is explicitly approved and documented. **Success:** no contradictory copy; no identifier is persisted or transmitted without an approved basis. **Verify:** focused component/privacy/copy/accessibility tests, browser print/copy smoke, `verify:cheap`, production-readiness. **Stop:** any new storage or provider transmission requires a separate review. | `src/components/patient-safety-plan.tsx:649`; `src/app/privacy/page.tsx:28`; `docs/privacy-impact-assessment.md` | 2026-07-24 | +| #053 | P1 | task | Close or explicitly defer the APP 8/DPA/ZDR governance basis | **Outcome:** counsel/account owners record an executed or explicitly deferred basis for OpenAI and Railway overseas processing, retention, and APP 5 wording. **Next:** execute or disposition both DPAs, ZDR, Australia residency, prompt-cache retention, subprocessors, and legal approval using the existing checklist. **Success:** every status field has dated evidence or explicit risk acceptance; PIA, notice, contracts, and provider configuration agree. **Verify:** counsel sign-off, provider readback, repository record. **Stop:** do not represent real-patient use as governance-cleared while open. | `docs/openai-cross-border-basis.md`; PIA-1/5/6; `docs/operator-backlog.md` | 2026-07-24 | +| #054 | P2 | issue | Missing answer relevance metadata is treated as source-backed | **Outcome:** absent `relevance` metadata renders conservatively. `RagAnswer.relevance` is optional, but `relevance?.isSourceBacked !== false` treats `undefined` as source-backed. **Next:** add the red render-policy test, then make the smallest policy-only fix. **Success:** missing and explicit-false relevance fail closed; explicit source-backed relevance is unchanged. **Verify:** answer-render-policy, provenance, clinical-safety, `verify:cheap`, production-readiness. **Stop:** do not expand into retrieval, ranking, or generation. | `src/lib/types.ts:1029`; `src/lib/answer-render-policy.ts:145` | 2026-07-24 | +| #055 | P2 | issue | Upload crash can strand a queued document without a job | **Outcome:** a crash between document and job creation cannot strand an upload indefinitely. **Next:** add the stranded-row reproducer, then choose the smallest idempotent atomic-enqueue RPC or bounded scheduled sweep consistent with current ownership and rollback contracts. **Success:** exactly one recoverable job is created; existing open jobs do not duplicate; owner scope, retry, audit, and rollback remain intact. **Verify:** focused upload/recovery/schema tests, migration guards, disposable replay, drift, `verify:cheap`, production-readiness. **Stop:** no at-least-once claim until the crash case passes; hosted changes require approval. | `src/app/api/upload/route.ts`; `docs/webhooks.md:168-192` | 2026-07-24 | +| #056 | P2 | task | Reconcile the preserved browser-readiness and contrast patch | **Outcome:** disabled Formulation navigation remains legible and Firefox/WebKit waits prove real React readiness without weakening assertions. **Next:** prove ownership of `agent/formulation-disabled-contrast`, compare its four-file diff with current `main`, retain only reproducing fixes, then run the affected specs in all browsers. **Success:** disabled contrast passes; a missing handler still fails; Chromium, Firefox, and WebKit pass. **Verify:** focused unit/Playwright, `verify:cheap`, identity-verified `ensure`, `verify:ui`. **Stop:** discard stale harness workarounds that no longer reproduce. | preserved worktree `release-browser-contrast-20260723`; session 2026-07-24 | 2026-07-24 | +| #057 | P2 | task | Prove one exact-SHA release candidate end to end | **Outcome:** local and hosted release evidence evaluates the same SHA and safely proves the production `RAG_QUERY_HASH_SECRET` boot requirement. **Next:** after candidate fixes land, obtain approvals, verify SHA/environment identity, confirm secret placement without revealing it, and run the bounded canonical release checks. **Success:** required gates are green and any waiver is explicit/owned. **Verify:** release runbook, golden retrieval, RAG-only quality, production-readiness, deployment/health identity, protected-main merge proof. **Stop:** no unidentified tree, duplicate paid canary, invented staging, or exposed secret. | `docs/operator-backlog.md`; `docs/launch-operator-runbook.md`; PIA-2 | 2026-07-24 | +| #058 | P2 | task | Compact document source text accordion is paused | **Outcome:** hand off the completed two-level closed accordion for every canonical `/documents/[id]` viewer without losing deep-link, search-highlight, print, mobile, or composer behavior. **Next:** only when the user explicitly returns to this task, prove the clean preserved branch `codex/chat-document-text-accordion-7cb4`, reconcile its two commits with current `main`, rerun focused document-viewer tests and the required local gates, then use the protected-main PR path. **Success:** Text, full page text, and passages start closed; navigation, citations, and search open the correct disclosure; nested rows are mutually exclusive; print restores state; 320/390/1280 px have no overflow or composer obstruction. **Verify:** focused mocked Playwright, `verify:cheap`, identity-verified `ensure`, `verify:ui`, static production-readiness. **Stop:** do not resume implementation, run its tests/server, or publish it without an explicit return to that paused task. | branch `codex/chat-document-text-accordion-7cb4`; commits `f09857da2`, `62521a86c`; delegation pause 2026-07-24 | 2026-07-24 | | #051 | P2 | task | Stabilise the live answer-quality canary before more RAG tuning | Diagnostics landed in PR #1095: structured JSON/Markdown artifacts now record the actual checked-out SHA, run identity and latency context, and the offline trend tool separates content, provider-route and latency outcomes. First validating run `30018289898` recorded the expected tree and cost, with 36/36 retrieval green, but one report cannot establish variability; PR #1097 prevents a single failure being mislabeled as repeated. Next: compare the scheduled 2026-07-26 structured report with this run. Do not spend on an immediate retry or reapply the archived lithium guard before that comparison. | PR #1095; run `30018289898`; PR #1097; archive ref `refs/archive/rejected-rag/20260723/monitoring-subject-gate` | 2026-07-23 | | #001 | P2 | task | Semantic reranking still gated off | `RAG_SEMANTIC_RERANK_ENABLED=false` from PR #901. Do not enable until the provider-backed 36/36 retrieval-quality gate **and** an ambiguity-focused canary are explicitly approved and recorded. | `docs/process-hardening.md` (Semantic reranking rollout debt); PR #901 | 2026-07-21 | | #005 | P3 | rec | `finalScore` saturates at clamp ceiling | Base + ~40 stacked boosts routinely exceed 1.0, so strong matches tie at 1.0 and order by an arbitrary `document_id` tiebreak. If ranking is ever revisited, break ties by the **pre-clamp** score rather than raising the `[0,1]` ceiling (downstream gates assume `[0,1]`). Ordering already sorts by the unbounded pre-clamp `rankScore` (`clinical-search.ts:1735,1927,1950-1955`), so the clamp confines only the reported confidence value, not result order. Not a defect on the current golden set; any change here is a protected RAG surface (canary required). | `docs/rag-hybrid-findings-and-todo.md` P1 item 4; `src/lib/clinical-search.ts:1735` | 2026-07-21 | @@ -56,7 +125,7 @@ Durable, cross-session memory of everything still outstanding for this repo: ope | #023 | P3 | task | Read Sunday 2026-07-26 scheduled-run artifacts | The 18:00 UTC scheduled runs deliver three free datapoints at once: first full-44 weekly canary (validates the #1044 ANSWER_CASE_LIMIT raise), browser-matrix flake second datapoint (webkit ui-route-coverage now reproduced + root-caused 2026-07-22 → see #024; firefox ui-formulation:91 still awaits a datapoint), and the irrelevant@10 labeling-audit artifact (§3.1 human-decision class). Read all three, then disposition. | sessions 2026-07-20/21; branch-review-ledger convergence notes | 2026-07-21 | | #024 | P3 | issue | WebKit e2e `_rsc`-prefetch access-control-checks errors | verify:release:offline on `main` ce32fe170 (2026-07-22) reproduced #023's webkit clause: **6/6 deterministic** failures in `tests/ui-route-coverage.spec.ts` (Therapy Compass; DSM home/comparison; Specifier comparison/map; Differential stream), each a `pageerror … ?_rsc=… due to access control checks` on Next.js RSC prefetch — Chromium + Firefox clean. Not merge-blocking (required gate `test:e2e:pr` is chromium-only; the full webkit matrix is advisory/release-time). Most likely a Playwright route-interception × WebKit interaction, not a Safari user defect. Next: decide (a) allow/mock the `_rsc` routes for the `webkit` e2e project, or (b) confirm real Safari impact — before trusting the full-matrix webkit gate at release. NB the 2 other webkit fails (`ui-stress:412`, `ui-universal-search:210`) passed on isolated re-run = true flake. | session 2026-07-22 (verify:release:offline, `main` ce32fe170); refines #023 | 2026-07-22 | | #025 | P2 | task | Activate the three webhooks (operator secrets) | Merged (#968) + deployed but inert — verified live: `POST /api/webhooks/railway` returns `503 webhook_not_configured`. To turn on: (1) Railway → set `RAILWAY_WEBHOOK_SECRET` + add the `?token=…` webhook URL; (2) the chat URLs `SLACK_WEBHOOK_URL`/`DISCORD_WEBHOOK_URL` must be set in BOTH places — the Railway **app/server env** (the receiver forwards deploy alerts via `postChatNotification`, which reads server env, so repo-secret-only leaves the Railway webhook authenticated but returning `delivered:false`) AND as **GitHub repo secrets** (the CI-failure workflow reads `secrets.*`); (3) `SUPABASE_INGESTION_WEBHOOK_SECRET`. Each fails closed until set, so this is pure ops. See docs/webhooks.md. | session 2026-07-22; PR #968; docs/webhooks.md | 2026-07-22 | -| #026 | P2 | task | Wire the Supabase document-change trigger | Implementation is complete on `codex/supabase-document-change-trigger`: forward migration `20260723150000_document_change_ingestion_webhook.sql`, schema mirror, update-only/minimal/fail-safe contract test and rollback docs. On 2026-07-24, `drift:manifest` replayed the full schema successfully in disposable Supabase Postgres 17.6.1.127, regenerated the manifest and removed the container; focused schema/drift tests passed 79/79 plus migration-role, function-grant and owner-scope guards. Next: protected-main PR and hosted migration-chain replay, then apply the committed migration through the normal Supabase path before configuring the Vault secret and base-URL GUC. The trigger remains inert until all three live steps are complete. | local branch/worktree; `docs/webhooks.md` section 3; session 2026-07-24 | 2026-07-22 | +| #026 | P2 | task | Activate the merged Supabase document-change trigger | Implementation and disposable replay are complete and merged to `main` by PR #1100: forward migration `20260723150000_document_change_ingestion_webhook.sql`, schema mirror, update-only/minimal/fail-safe contract test, drift manifest, and rollback docs. The trigger remains inert. Next: after #055 settles the complementary INSERT-recovery contract, obtain explicit Supabase approval, run the hosted migration-chain replay, apply the committed migration through the normal path, configure the Vault secret and base-URL GUC, and verify a controlled update event. Success: eligible false-to-true updates enqueue once, INSERT remains outside this trigger, failures stay fail-safe, and rollback/readback evidence is recorded. Stop: never apply raw SQL live or configure secrets without named ownership and approval. | PR #1100; `docs/webhooks.md` section 3; session 2026-07-24 | 2026-07-22 | | #027 | P3 | rec | External uptime monitor independent of GitHub/Railway | `live-domain-monitor.yml` runs on GitHub's cron, so it won't run in exactly the outage it should catch (Actions or the deploy itself down). Add an off-platform synthetic monitor (UptimeRobot / Better Stack / Checkly) hitting `/api/health` with a webhook alert. Provider setup, not code. | session 2026-07-22 webhook review | 2026-07-22 | | #028 | P3 | rec | Runtime error tracking (Sentry or similar) | No error tracking in the repo — production exceptions on `psychiatry.tools`, including how often `RAG_PROVIDER_MODE=auto` silently degrades to source-only, are invisible. Weigh adding `@sentry/nextjs` (dependency + DSN secret + instrumentation) vs cost; alert → chat/issue. Provider-backed; needs explicit sign-off before adding the dependency. | session 2026-07-22 webhook review | 2026-07-22 | | #029 | P2 | issue | 12 of 30 answer-quality cases return the fallback stub | run #61 --dump-answers: 12/30 quality cases emit the source_backed_review_fallback boilerplate with answer_sections: [], all grounded with 4-6 citations. Some still PASS targeting because the stub echoes query keywords (the contraindication/document_lookup matchers need only a keyword), so the targeting metric MASKS the problem for those intents. Superset of #018 — fix in the extractive composer, validate with the provider-backed answer eval. | run #61 dump artifact; session 2026-07-22 | 2026-07-22 |