feat(db): migration-history probe, guard-migration contract, and index-monitoring ratchet (remediation Phase 6)

[BigSimmo]

Summary

Remediation plan Phase 6 (docs/database-remediation-plan.md §6.1–6.3), built repo-side only. Worker chat with no owned ledger row; #316 and #056 were not touched.

• 6.1 Migration-history probe. New migration supabase/migrations/20260818090000_schema_drift_snapshot_history_probe.sql redefines public.schema_drift_snapshot() (v2): same inventory plus migration_history — every supabase_migrations.schema_migrations version whose statements is NULL or empty (the mark-applied / history-repair fingerprint from forensics §1.1) — and migration_history_probe (ok / no_history_table / no_statements_column, never a silent empty array). Read via a guarded dynamic query so a replay without the history schema still succeeds. Mirrored byte-identically into supabase/schema.sql; supabase/drift-manifest.json regenerated from the Docker replay (the new body executed there: probe no_history_table, snapshot_version 2). scripts/check-drift.ts never compares this category manifest-vs-live; each live row is reported as ! [migration_history] no_statements <version> unless a structurally valid migration_history allowlist entry covers it, and a live snapshot without the probe prints an info line naming the pending deploy.
• This migration is NOT deployed. It needs the owner-approved production migration deploy window named in the plan's approval map (Phase 6.1 — "one migration deploy, yes for 6.1"), scheduled after Phase 4. Until it is applied, the weekly live-drift run will additionally show a mismatch public.schema_drift_snapshot() function finding (repo-ahead body) — that is the pending deploy, not a body regression, and the script says so.
• 6.2 Guard-migration contract. Documented in docs/database-drift-detection.md ("Migration-history probe", "Guard-migration contract") and AGENTS.md ("Supabase project safety"): any mark-applied / history repair MUST ship a fail-fast validation migration per 20260804110240. supabase/drift-allowlist.json migration_history entries carry reason + guard {class, migration, objects}; classes validation (mandatory for versions from 2026-08-18), superseded, no_ddl (pre-contract history only). check:drift enforces the structural half at runtime (a malformed entry never silences a row); tests/migration-history-guards.test.ts verifies each guard file really covers its listed objects. Seeded five §1.1 versions with repo-provable superseded guards (20260701010000, 20260701020000, 20260701030000, 20260701060000, 20260702000000). The remaining §1.1 rows and the 2026-07-12 batch are deliberately not allowlisted — they will be the expected findings of the first post-deploy run and need real guard migrations, not bare allowlist entries.
• 6.3 Runtime index-monitoring ratchet. tests/search-health-index-coverage.test.ts + supabase/search-health-unmonitored-indexes.json: every repo-defined index (migration replay ∪ manifest; *_pkey exempt) on documents, document_chunks, document_index_units, document_embedding_fields, document_memory_cards, rag_retrieval_logs must be in search_schema_health() required_indexes/index_aliases (parsed from the latest definer, cross-checked with schema.sql) or listed with a reason and disposition. Seeded 44 entries, 8 monitor-candidate including the three §1.3-absent indexes on those tables (document_chunks_anchor_idx, document_index_units_heading_path_idx, documents_registry_projection_lookup_idx); the other 17 absent indexes are on tables outside this scope. The test failed with exactly those 44 names before the list existed.
• Residual queued via npm run issues:add (deploy window, first-run triage, guard migrations for the pre-contract rows, Phase 4.4 decision on the monitor candidates). Forensics file gained a Phase 6 section.

RAG impact: no retrieval behaviour change — observability-only snapshot extension and repo-side tests.

Verification

• npm run verify:pr-local — every gate green except test, which failed only on tests/session-start-hook.test.ts (5 tests, exit 127 head: command not found in this Windows Git Bash plus EPERM on a temp dir); reproduced alone with the same environmental cause and unrelated to this diff. Full suite otherwise: Test Files 1 failed | 641 passed | 2 skipped (644), Tests 5 failed | 6861 passed | 27 skipped (6893); lint, typecheck, check:pr-policy, check:ledger-write-discipline, docs:check-links (1838 repo path references resolve) all passed. The three not-reached checks were run directly: Offline RAG fixture and manifest validation passed (36 golden cases, 25 suites)., data/medication-interaction-index.json is up to date (523 rows)., docs/medication-interaction-lexicon-review.md is up to date (28 catalogue terms).
• npm run check:migration-role — Hosted migration-role guard passed: active hosted SQL/tooling uses postgres and immutable applied history is unchanged.
• npm run drift:manifest — Replay complete in 46s, Wrote supabase/drift-manifest.json (Docker, digest-pinned image, --pull=never).
• Focused schema/drift suite: tests/drift-detection.test.ts tests/supabase-schema.test.ts tests/migration-history-guards.test.ts tests/search-health-index-coverage.test.ts tests/live-drift-workflow.test.ts tests/migration-history-placeholders.test.ts tests/forward-codify-retrieval-targets.test.ts tests/hosted-migration-role-guard.test.ts — Test Files 8 passed, 127 passed.
• Red-then-green proofs: ratchet test with an empty list → 44 index(es) on retrieval-critical tables are neither in search_schema_health() required_indexes/index_aliases (20260706010000_search_schema_health_m13_guard.sql) nor listed …; guards test with one entry pointed at a nonexistent guard → allowlist entry 20260701030000 (superseded → 20260706010000_does_not_exist.sql): guard file missing; both green after restore.
• npm run verify:ui — not applicable (no UI change).
• npm run verify:release — not run (no release claim).
• npm run eval:retrieval:quality — not applicable (no retrieval, ranking, selection, chunking, or scoring change; no src/lib/rag/** or match_* body touched).
• npm run eval:rag / npm run eval:quality — not applicable (no answer-generation change).
• Verification not run: npm run check:production-readiness and npm run check:drift against live — provider-backed; this task was authorised as repo-side only with no hosted access. check:drift against live is what the deploy window will run.
• npm run check:deployment-readiness — not applicable.

Risk and rollout

• Risk: low on the repo side. The migration is additive/observability-only (same inventory keys, snapshot_version 1→2, two new keys) and read-only; check-drift.ts changes only add a category, so existing object comparisons are byte-for-byte unchanged (existing unit tests still pass). Two new tests are ratchets: they will fail future PRs that add an unmonitored index on a retrieval-critical table or a migration_history allowlist entry without a real guard — that is the intent. Once deployed, check:drift will report roughly nine §1.1 versions plus the six 2026-07-12 batch versions as unguarded no_statements findings on top of today's red state; the follow-up (queued) is to author validation guard migrations for them.
• Rollback: revert this commit (single commit); if the migration has been deployed, redeploying the previous 20260706200000 body via a new migration restores v1 (the function is create or replace, no data). Deleting the two JSON files and tests removes the ratchets.
• Provider or production effects: None in this PR. The 6.1 migration is authored but not deployed; deployment requires the owner-approved production migration window (plan approval map Phase 6.1, after Phase 4). No Supabase, OpenAI, or CI call was made in this session; the manifest was regenerated in a local Docker container.

Clinical Governance Preflight

Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.

• Source-backed claims still require linked source verification before clinical use
• No patient-identifiable document workflow was introduced or expanded without explicit governance approval
• Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
• Service-role keys and private document access remain server-only
• Demo/synthetic content remains clearly separated from real clinical sources
• Source metadata, review status, and outdated/unknown-source behavior remain conservative
• Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

• Flagged before editing: touches supabase/migrations/** + supabase/schema.sql (operational-risk surface, migration replay gates apply) and reads search_schema_health(); does not touch src/lib/rag/** or any match_* body.
• Open-PR check for schema_drift_snapshot / check-drift / drift-allowlist / search_schema_health / drift-manifest surfaces before starting (#292): none open. docs/database-remediation-coordination.md is not on main yet (open PR docs(db): land and re-baseline the remediation coordination board (#316) #2044) and was read from that head.
• The coordination board (PR docs(db): land and re-baseline the remediation coordination board (#316) #2044) lists Phase 6 as "can be designed repo-side any time; deploy after Phase 4" — this PR is that repo-side design and build.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[supabase]

Updates to Preview Branch (claude/database-drift-allowlist-48839e) ↗︎

Deployments	Status	Updated
Database	✅	Mon, 17 Aug 2026 17:29:24 UTC
Services	✅	Mon, 17 Aug 2026 17:29:24 UTC
APIs	✅	Mon, 17 Aug 2026 17:29:24 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Mon, 17 Aug 2026 17:29:26 UTC
Migrations	✅	Mon, 17 Aug 2026 17:29:28 UTC
Seeding	✅	Mon, 17 Aug 2026 17:29:29 UTC
Edge Functions	✅	Mon, 17 Aug 2026 17:29:30 UTC

---

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 52 minutes

Limit details: You’ve used all 1 included review currently available under your plan. You completed 95 included PR reviews in the past 7 days; at that activity level, included reviews refill at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d2a9e1a6-580a-4e12-85b6-2ffde4cc6cdf

📥 Commits

Reviewing files that changed from the base of the PR and between 5fd8649 and 2963812.

📒 Files selected for processing (14)

• AGENTS.md
• docs/audit/live-drift-forensics-2026-08.md
• docs/branch-review-records/014fee0f5a9c0fc049bfa0047afa855ccd498160d291ebb5c3d2eb0b0749a7cf.record.md
• docs/database-drift-detection.md
• docs/outstanding-issues-inbox/d6ce8a1d-518d-48fc-8a9a-7796f060a46e.json
• scripts/check-drift.ts
• supabase/drift-allowlist.json
• supabase/drift-manifest.json
• supabase/migrations/20260818090000_schema_drift_snapshot_history_probe.sql
• supabase/schema.sql
• supabase/search-health-unmonitored-indexes.json
• tests/drift-detection.test.ts
• tests/migration-history-guards.test.ts
• tests/search-health-index-coverage.test.ts

---

_{Comment @coderabbitai help to get the list of available commands.}