fix(rag): retry unverified numeric answers safely

[BigSimmo]

Summary

• Treats deterministic unverifiedNumericTokens as the specific provider-safe reason numeric_faithfulness_gap rather than flattening the failure into a generic generation-quality token.
• Feeds that signal into the existing fast-to-strong retry with bounded instructions requiring exact source digits and units.
• Does not add a provider attempt, extend any timeout, weaken a quality gate, alter retrieval or ranking, or change the conservative extractive fallback.
• Adds allowlisted failure parsing plus focused regression coverage for an unsupported 500 mg fast answer repaired to the cited 250 mg.

Conflict-resolution note

Current main was merged at a5f03ccc3390803a2b9de1d91701cf3a9d0455a8 in commit 4760b5e8c0d2a41043a7a947555e40bd9255cccd. The canonical docs/branch-review-ledger.md and docs/outstanding-issues.md mutations were deliberately not replayed over newer repository history. The post-resolution diff contains five implementation and test files; task #231 remains open and current-main ledger state remains authoritative.

RAG impact: no retrieval behaviour change. This modifies generation-quality classification and the existing fast-to-strong synthesis retry only. Retrieval, ranking, selection, ordering, provider-attempt count, route budgets, caching, and fallback authority are unchanged.

Verification

• Prior identical implementation: tests/rag-generation-failure.test.ts and tests/rag-answer-fallback.test.ts passed 93/93 tests.
• Prior identical implementation: offline RAG validation passed 23/23 files, 575/575 tests, and 36 golden cases.
• Prior current-main-based checks: lint and typecheck passed. Formatting and repository ledger guards passed before the stale canonical-ledger files were removed from the final diff.
• Fresh conflict check: GitHub reports head 4760b5e8c0d2a41043a7a947555e40bd9255cccd mergeable against current main.
• Fresh hosted checks: Secret Scan passed. SAST and CI are running on the conflict-resolution head.
• Full release verification and provider-backed evaluations were not run. This remains a draft PR and no release-confidence claim is made.

Risk and rollout

• Risk: This changes answer-generation behaviour when a fast model introduces an unverified number. The strong retry receives stricter source-verbatim numeric instructions. If generation still fails or times out, the existing grounded extractive fallback remains authoritative.
• Rollback: revert this PR. The independently merged generation-fallback cache exclusion remains unaffected.
• Provider or production effects: none during this conflict-resolution sweep.

Clinical Governance Preflight

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo and synthetic content remain separated from real clinical sources
• Source metadata, review status, and unknown-source behaviour remain conservative
• Deployment classification and TGA SaMD impact were checked

Notes

This remains a draft because the deterministic repair is proven offline, while the normal-budget live canary timed out before model synthesis and did not validate the repaired live synthesis path. Do not repeat the unchanged canary, increase route or provider timeouts, add provider attempts, weaken numeric-quality gates, or cache generation fallbacks.

[supabase]

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.

---

Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a6fd0560-29b0-4309-88fc-50e7d09a7b88

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[BigSimmo]

Adjudication against PR #1899 (claude/lithium-generation-quality-debug-ji1vce), per the operator's request on ledger #231:

This draft is superseded for Phase 1 and is being closed. Rationale:

• Its diagnostic half (surfacing the specific quality-gate reason instead of the flattened generation_quality_failed token) is covered by feat(rag): record the structured generation-quality verdict on fallback (#231 phase 1) #1899, which does it without changing the degraded token the UI, cache-exclusion, and SLO probes see — this draft's generation_quality_failed_<reason> suffix changes that contract, while feat(rag): record the structured generation-quality verdict on fallback (#231 phase 1) #1899 rides the verdict alongside in answer_retry_reasons and dedicated log fields.
• Its behaviour half (promoting numeric_faithfulness_gap into the fast-gate classifier plus a stricter numeric-grounding retry instruction) is a genuine Phase 2 candidate, but fix(ci): bump checkout/setup-node/cache actions to v5 #231's protocol is instrumentation first, then a bounded fix justified by probe evidence with its own live canary pair. This draft's normal-budget canary timed out before validating the repaired synthesis path, verify:pr-local did not complete on the current head, and the branch is merge-dirty against main.
• Nothing is lost by closing: the retry design here should be re-cut on a fresh branch once the feat(rag): record the structured generation-quality verdict on fallback (#231 phase 1) #1899 instrumentation names the binding gate for live lithium traffic. If the probe confirms numeric_faithfulness_gap is the dominant reason, this draft's approach is the natural starting point and can be re-opened or re-cut with credit.

---

Generated by Claude Code