Skip to content

Fix answer search: port PR #315 Supabase-key fallback to /api/answer/stream + memoize classifier verdicts - #326

Merged
BigSimmo merged 3 commits into
mainfrom
claude/answer-search-diagnosis-8k6oy9
Jul 6, 2026
Merged

Fix answer search: port PR #315 Supabase-key fallback to /api/answer/stream + memoize classifier verdicts#326
BigSimmo merged 3 commits into
mainfrom
claude/answer-search-diagnosis-8k6oy9

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Diagnosis: the answer-search UI calls POST /api/answer/stream, but PR Fix public answer search when anonymous limiter is unavailable #315 ("fix public answer search") added the Supabase API-key configuration fallback only to /api/answer and /api/search. So after Fix public answer search when anonymous limiter is unavailable #315, document search recovered from a misconfigured/rotated Supabase key while answer search kept failing with a generic "Answer generation failed" stream error — exactly the reported "answer search is not working".
  • Port the Fix public answer search when anonymous limiter is unavailable #315 fallback to the stream route: outside production, a Supabase "unregistered/invalid API key" error now degrades to a visible demo answer (fallbackMode: "non_production_demo") instead of an SSE error. The route's inline demo payload builder was extracted into buildDemoAnswerPayload() and reused for both the demo branch and the fallback.
  • In production (where the demo fallback is deliberately inert), the SSE error payload now carries a stable details.code = "supabase_api_key_configuration" so operators can diagnose a bad deploy key from the browser network tab.
  • Apply the documented finding Improve Clinical KB dashboard and RAG hardening #11 interim hardening (docs/process-hardening.md): memoize the generative query classifier's definitive verdicts per normalized query (10 min TTL, 500-entry cap) in src/lib/rag.ts, so bare low-confidence queries ("bipolar disorder") stop nondeterministically flipping between answered and unsupported_short_circuit. Transient failures (timeout/parse error) are NOT cached, mirroring the rag_aliases lesson. The soft-tail short-circuit thresholds are untouched per the 2026-07-03 decision.
  • Tests: two new stream-route cases in tests/private-access-routes.test.ts (fallback outside production; tagged error, no fallback in production — the fallback test fails against the unfixed route) and new tests/rag-classifier-memo.test.ts (decline memoized; transient failure retried).

Operator note (production): if the deployed site still fails, the live environment's Supabase keys are the remaining suspect — verify NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY (sb_publishable_…) and SUPABASE_SERVICE_ROLE_KEY (sb_secret_…) target project sjrfecxgysukkwxsowpy and run npm run check:supabase-project. Answer-search SSE errors with code supabase_api_key_configuration confirm this case.

Verification

  • npm run verify:cheap — green (lint 0 errors, typecheck, 1101 vitest tests, runtime + sitemap checks)
  • npm run verify:ui — green (103 Chromium tests, including all 9 answer-flow smoke tests)
  • npm run verify:release — not run (release gate; needs live Supabase/OpenAI keys unavailable in this environment)
  • npm run format:check
  • npm run eval:retrieval:quality — not triggered: no retrieval selection/scoring behavior changed (classifier memoization only makes the existing verdict deterministic per query; thresholds and ranking untouched). Needs live keys, cannot run in this environment — flagging per the template rule.
  • npm run eval:rag / eval:quality --rag-only — not triggered: synthesis prompt and answer post-processing unchanged
  • npm run check:production-readiness — only the expected missing-secret failures of this demo-mode environment (no new findings)
  • npm run check:deployment-readiness — not applicable: no deployment/startup behavior changed

Also verified end-to-end in demo mode: POST /api/answer/stream returns progress + final demo answer with citations, and the new fallback test fails against the pre-fix route (red) and passes with the fix (green).

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use — answer/refusal contracts unchanged
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — no env/project changes
  • Service-role keys and private document access remain server-only — error payload exposes a stable code only, never key material
  • Demo/synthetic content remains clearly separated from real clinical sources — fallback answers are the existing synthetic demo corpus, tagged demoMode: true + fallbackMode: "non_production_demo", and production never serves them
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative — source-governance refusal path untouched
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed — no clinical decision-support behavior change; failure-path handling and determinism only

Notes

  • Environment caveat from this run: npm run ensure cannot work in IPv6-less containers (scripts/dev-free-port.mjs requires binding ::1/::, so it exhausts all ports). Worked around locally by launching Next directly on the stable managed port; possibly worth a follow-up to make the IPv6 probe conditional.

🤖 Generated with Claude Code

https://claude.ai/code/session_0179yo6uA4zAxx2hMKS88MXG


Generated by Claude Code

…oute

The UI's answer search calls POST /api/answer/stream, but PR #315's
Supabase API-key configuration fallback only landed on /api/answer and
/api/search — so document search recovered while answer search kept
failing with a generic stream error. Port the non-production demo
fallback to the stream route (extracting its inline demo payload
builder for reuse) and tag key-configuration errors with a stable
`supabase_api_key_configuration` code in the production SSE error
payload so operators can diagnose it from the client.

Also apply the documented finding #11 interim hardening: memoize the
generative query classifier's definitive verdicts per normalized query
(10 min TTL) so bare low-confidence queries like "bipolar disorder"
stop nondeterministically short-circuiting to "unsupported" between
runs. Transient classifier failures are not cached, mirroring the
rag_aliases lesson. No retrieval selection/scoring behavior changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179yo6uA4zAxx2hMKS88MXG
@supabase

supabase Bot commented Jul 6, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

Confirmed live: the project's legacy anon/service_role JWTs were
disabled on 2026-07-05, and PostgREST rejects them with "Legacy API
keys are disabled" — which the existing unregistered/invalid matcher
missed, so neither the non-production demo fallback nor the tagged
stream error code fired for the exact failure breaking answer search.
Match that message (and "Secret API key required") in
isSupabaseApiKeyConfigurationError, with unit coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179yo6uA4zAxx2hMKS88MXG
@BigSimmo
BigSimmo marked this pull request as ready for review July 6, 2026 13:45
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@BigSimmo
BigSimmo enabled auto-merge July 6, 2026 13:45
…ier memo

Main landed a parallel fix for the same answer-stream gap (PR #329-era
buildDemoStreamAnswer + classifier verdict memoization). Resolution keeps
main's implementations wholesale (its memo adds in-flight coalescing and an
analyzer-versioned cache key) and drops this branch's duplicates, while
preserving this branch's additions main lacked: the stable
supabase_api_key_configuration code on production stream errors and the
expanded key-error detection ("Legacy API keys are disabled" — confirmed
live) with its unit tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0179yo6uA4zAxx2hMKS88MXG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants