feat(rag): intent-conditioned related-information menu and moderate answer length (packet S2 / A2 + A3)

[BigSimmo]

Summary

Packet S2 of the RAG improvement programme (docs/rag-improvement/HANDOVER.md §3, README §A2 + §A3): intent-conditioned related-information composition plus the moderate answer-length increase. Both are prompt-surface changes and share one canary pair, so they ship together (the combined diff stayed reviewable — S2b is not needed).

RAG impact: behaviour change — canary pair 32100681177 -> post-merge eval-canary dispatch (owner-approved; the baseline run is the S1d confirmation run on 4ea310e48, this PR's merge base)

• A2 — composition menu. New pure module src/lib/rag/answer-composition.ts maps (RagQueryClass, ClinicalQueryIntent) to a related-information menu — the answerSections kinds a psychiatrist colleague would append unprompted for that question type — and buildAnswerInput (src/lib/rag/rag.ts) serialises it as one related_information_menu: line in the "Interpreted clinical task" block, keyed on the final queryClass and queryAnalysis.intent. answerInstructions gains one bullet under "Answer sections" describing the menu as advisory, evidence-gated, cited like any other section, and subordinate to the verbatim narrow-question rule. No new pipeline stage, RagAnswer field, or render block; grounding, verification, claim support, routing, retrieval, ranking, selection, and the render trust ladder are untouched.
• A3 — moderate length. Prompt targets move from "1-3 short sentences, about 35-75 words" to "2-4 sentences, about 60-110 words" for complex questions, and from "two to five" to "three to six" sections when the excerpts support them; the narrow-question rule (- If the question is narrow (a definition, one threshold, a single dose, a yes/no), answer only that. …) is byte-identical, and the length line restates that a narrow question keeps 1-3 sentences / 35-75 words. The structured-output schema's answerSections.maxItems rises 5 → 6 so the decoder allows what the prompt now asks for.
• Version roll. ragAnswerPromptVersion v18 → v19 and ragAnswerSchemaVersion v3 → v4 (src/lib/rag/rag-versioning.ts) so rag_response_cache and the OpenAI prompt cache roll; the provider wrapper's fallback prompt_cache_key (src/lib/openai.ts) and its pin in tests/openai-cache.test.ts move in lockstep, as the v17→v18 commit did.
• Baseline re-capture. check:rag:adversarial-fixtures cross-checks the baseline record's promptVersion against the live prompt, so scripts/fixtures/rag-adversarial-baseline.v1.json was re-recorded against the evaluated code commit b7aa925f0ae19e89a9f0acf842b4a80d84083fb5 (v19, index_version 20260818090000_schema_drift_snapshot_history_probe, offline gates re-run; provider gates stay pending_owner_run with run 32100681177 as priorRun). docs/rag-improvement/baseline-record.md §4 explains it.
• Docs and ledger. HANDOVER §2 S2 row opened, S1d row corrected to canary pair 32052479537 -> 32100681177 green, S2b marked not needed, S3 precondition updated; README §2 answer-shape anchor updated; docs/rag-behaviour/behaviour-map.md §8 added; the two stale planned-path entries in scripts/check-docs-links.mjs removed (S4 precedent); tests/answer-composition.test.ts added to the offline RAG contract list (26 suites); an /issues add inbox request queued for the ci-change-scope.mjs rag_eval_changed regex gap on src/lib/rag/** (own PR, not bundled).

Menu table (class is authoritative; intent refines only the two clinical-fact classes)

queryClass	intent	Menu (kind — focus, in order)
medication_dose_risk	any except escalation_risk	monitoring_timing — monitoring schedule and levels; contraindications_cautions — contraindications and cautions; escalation_risk — escalation and stop triggers; medication_dose — dose adjustment for renal or hepatic impairment and older adults
medication_dose_risk / table_threshold	escalation_risk	required_actions — immediate actions; thresholds — the thresholds that trigger them; escalation_risk — who to contact or refer to; documentation — what to document
table_threshold	any except escalation_risk	thresholds — adjacent thresholds or bands in the same scale; required_actions — required actions per band; escalation_risk — escalation pathway
comparison	any	comparison — decision factors; comparison — per-source differences and conflicts; required_actions — switching or washout considerations
broad_summary	any	required_actions — weighted management map: risk, first-line, adjuncts, monitoring, special populations; documentation — documentation and forms; source_gap — source gaps
document_lookup, unsupported_or_general	any	none — related_information_menu: none — no related-information menu for this question type; apply the Answer sections rules as written

Design notes, each verified against source: (1) intentFromSignals (src/lib/clinical-search.ts) matches "long-term" / "determine" as definition, so definition does not silence a menu — narrowness is the prompt rule's job; (2) unsupported_or_general is the only class where isOverExpandedSimpleGeneratedAnswer (> 95 words / > 1 section) can fire, so it carries no menu; (3) README's "related documents" item is not offered — relatedDocuments is deterministic (buildRelatedDocumentsSafe) and already renders at high trust; (4) queryClass can be overridden by mode/classifier while queryAnalysis.intent stays heuristic, so the map is total over all 48 class×intent cells (pinned by test).

Prompt diff (src/lib/rag/rag-answer-instructions.ts)

-- Plain prose, usually 1-3 short sentences, about 35-75 words. The FIRST sentence must be complete and must directly answer the question; lead with the answer, then only the vital supporting detail.
+- Plain prose, usually 2-4 sentences, about 60-110 words. The FIRST sentence must be complete and must directly answer the question; lead with the answer, then the vital supporting detail. A narrow question (a definition, one threshold, a single dose, a yes/no) still gets a narrow answer of 1-3 short sentences, about 35-75 words — the extra length belongs to management, medication, threshold-band, comparison, and multi-document questions where the evidence carries real yield, never to padding.
…
-… Complex clinical, medication, threshold, comparison, or multi-document questions: return two to five distinct sections when supported.
+… Complex clinical, medication, threshold, comparison, or multi-document questions: return three to six distinct sections when the excerpts support them; never pad to reach a count.
+- The "Interpreted clinical task" block carries a related_information_menu line: the related, high-yield section kinds a psychiatrist colleague would append unprompted for this question type, each written as kind — focus. Attempt those kinds, in that order, only when the retrieved excerpts directly support them; omit any menu item the excerpts do not support, silently, without a placeholder or a source-gap note about it, and never invent a section to fill the menu. Every menu section carries citation_chunk_ids like any other section. When the menu says none, add no related sections. The menu never overrides the narrow-question rule above: a single dose, threshold, or definition still gets a narrow answer.

rag.ts: +import { relatedInformationMenuLine } from "@/lib/rag/answer-composition";, +relatedInformationMenuLine(queryClass, queryAnalysis.intent), in interpretedTask (after answer_scope), and maxItems: 5 → 6 — rag.ts sits at exactly 4362/4362 lines of its no-growth budget.

trustCaps check (README §A2 asks for it)

trustCaps in src/lib/answer-render-policy.ts has no section-count key ({sources, rows, quotes, visual, related}); rows (6 medium / 8 high) caps evidence-map rows keyed by primary sources and related (4, high only) caps relatedDocuments. answerSections render uncapped in ClinicalDashboard.tsx (safeAnswerSections), and neither Zod, sanitizeAnswerSections, nor answer-stream-contract.ts caps the count — so no cap change is needed. Two accepted residuals: src/lib/ward-output.ts keeps its 5-row structured-support table (a 6th section still renders in the sections list), and the summary route shares answerJsonOutputSchemaForResults, so its ceiling also becomes 6 while its own prompt (ragSummaryPromptVersion) is untouched.

Verification

• npm run verify:pr-local — heavy scope selected (runtime, lock parity, format:changed, sitemap/docs/ledger checks, lint, typecheck, full unit suite, build, eval:rag:offline, eval:rag:adversarial:offline, medication checks): every stage green through lint and typecheck; npm run test → Test Files 1 failed | 651 passed | 2 skipped (654) / Tests 1 failed | 7023 passed | 27 skipped (7051) — the one failure is tests/session-start-hook.test.ts > still writes the env file when Claude Code provides one (Linux /tmp vs Windows temp path shape), reproduced identically at the merge base 4ea310e48 in a detached worktree (Tests 1 failed | 4 passed (5)), so it is a pre-existing Windows-host flake, not this diff. Because verify:pr-local stops at the first failing stage, the remaining selected stages were run directly and are green: npm run build → ✓ Compiled successfully in 66s / Client bundle secret surface check passed. (exit 0); check:medication-interactions → data/medication-interaction-index.json is up to date (525 rows).; check:medication-lexicon-report → docs/medication-interaction-lexicon-review.md is up to date (37 catalogue terms).; eval:rag:offline and eval:rag:adversarial:offline as below.
• Focused: npx vitest run tests/answer-composition.test.ts tests/rag-answer-composition-prompt.test.ts tests/rag-answer-fallback.test.ts tests/openai-cache.test.ts → Test Files 4 passed (4) / Tests 122 passed (122)
• npm run check:maintainability-budgets → [maintainability] src/lib/rag/rag.ts: 4362/4362 lines
• npm run check:rag:fixtures → Offline RAG fixture and manifest validation passed (36 golden cases, 26 suites).
• npm run eval:rag:offline at b7aa925f0 → Test Files 26 passed (26) / Tests 623 passed (623) — Offline RAG fixture and production-contract checks passed.
• npm run eval:rag:adversarial:offline → Adversarial fixture contract passed (24 synthetic cases, 8 categories, 6 canaries). / Tests 25 passed (25) — the three S5 KNOWN_DIVERGENCES pins are unchanged (the harness fails with "known divergence resolved" if one flips; none did).
• npm run check:production-readiness → PASS 2 (Node 24.19.0; query-hash boot guard); the FAIL rows are the missing provider env in this worktree (no .env.local: NEXT_PUBLIC_SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, OPENAI_API_KEY) — a capability gap of the offline session, not a regression.
• node scripts/check-docs-links.mjs → docs link check passed: 1867 repo path references resolve. (after removing the two stale allowlist entries)
• UI verification not run: no UI, routing, styling, or browser behaviour changed (prompt, pure module, versions, tests, docs).
• npm run eval:retrieval:quality not run: retrieval/ranking/selection untouched; 36/36 is covered by the post-merge canary dispatch below.

Requested, not executed (provider-backed; owner approves each):

1. Post-merge eval-canary dispatch — the post half of canary pair 32100681177 -> <post>; gates recall 1.0/1.0, zero per-case rr regressions, answer gate 44/44.
2. npm run eval:answer-quality — the 30 answerQualityEvalCases before/after comparison (Gate E fixed base). Known metric confound: scoreAnswerQualityEvalCase scores readability over the answer plus every section body with a 220-word ceiling (src/lib/rag/rag-eval-cases.ts); the S2 targets can exceed it by design, so a readability=0 flag driven only by total length is an artefact to adjudicate (raise the ceiling with an eval_config_version bump, or accept), not a regression. The scorer was left untouched here so the comparison runs under one definition — your call after seeing the distribution.
3. ~10 owner-chosen live questions (Gate E) — suggested spread: two dosing (medication_dose_risk × drug_dosing), two risk/withhold (× escalation_risk), two threshold-band, one comparison, one management summary, one document lookup, one bare definition (should stay narrow). Watch for: unsupported menu sections (should be absent, not padded), trust flips to low from a single unverified numeric token in a new section (hides quotes/related), and any overexpanded_simple_answer fallbacks (should not occur — the class carrying that gate gets no menu).

Risk and rollout

• Risk: answer composition and length change on every generated answer (fast and strong routes; extractive/source-only fallbacks are untouched). Failure still degrades conservatively — unsupported menu sections are dropped by sanitizeAnswerSections / claim support, and the finalizer's S1d extractive recovery still applies. Longer output spends more of the route budget; S1b (dosing → strong before the deadline) and S1d (final-gate gap recovery) landed first for exactly this reason. Live behaviour is only trusted after the canary pair and the Gate E comparison.
• Rollback: revert this PR (single squash commit); ragAnswerPromptVersion v19 isolates the caches, so a revert to v18 serves no S2-shaped cached answers. No migration, index, env, or dependency change.
• Provider or production effects: None from this PR itself. Post-merge canary dispatch and eval:answer-quality are requested above and require explicit approval; no provider call was made in this session.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Protected surface flagged before editing (session first message). Not touched: rag-routing.ts, rag-claim-support.ts, rag-extractive-answer.ts (finalizer), answer-render-policy.ts, smart-rag-api.ts, types.ts, clinical-search.ts, rag-eval-cases.ts, ward-output.ts, ci-change-scope.mjs, every retrieval/ranking/selection file, and the S5 KNOWN_DIVERGENCES. Condition-first for/in binding (fix(rag): bind claim-leading for/in conditions in the high-risk trigger check (S1c follow-up) #2065, reverted by revert(rag): back out #2065 claim-leading condition binding after live canary regression #2088) is not reintroduced.
• Ledger record appended with npm run ledger:append (immutable record, no table edit). HANDOVER §2 S2 row carries the PR number.
• Next packet: S3 (A4 follow-up refinement) consumes buildRelatedInformationMenu from src/lib/rag/answer-composition.ts; blocked on this merge + its green canary pair.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 12 minutes

Limit details: You’ve used all 1 included review currently available under your plan. You completed 101 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: e22d8d55-1cee-4131-9f59-487032211853

📥 Commits

Reviewing files that changed from the base of the PR and between 9d83245 and 3196f1e.

📒 Files selected for processing (18)

• docs/branch-review-records/7be5b09c1f8088555adaf8667b61b3bc5fc44e0e4504164733179aac5eb61df0.record.md
• docs/outstanding-issues-inbox/a727ac1a-1d72-41bd-88f9-76945528bc97.json
• docs/rag-behaviour/behaviour-map.md
• docs/rag-improvement/HANDOVER.md
• docs/rag-improvement/README.md
• docs/rag-improvement/baseline-record.md
• scripts/check-docs-links.mjs
• scripts/fixtures/rag-adversarial-baseline.v1.json
• scripts/fixtures/rag-offline-contract-tests.json
• src/lib/openai.ts
• src/lib/rag/answer-composition.ts
• src/lib/rag/rag-answer-instructions.ts
• src/lib/rag/rag-versioning.ts
• src/lib/rag/rag.ts
• tests/answer-composition.test.ts
• tests/openai-cache.test.ts
• tests/rag-answer-composition-prompt.test.ts
• tests/rag-answer-fallback.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.