harden(api): validate inbound DB and RPC payloads against Zod row contracts

docs/branch-review-records/66904e0507ba08c4f28e5760ba2b025ce41fac8f096ab7ba1cc4ef61db2eb599.record.md

@@ -0,0 +1 @@
+| 2026-08-17 | https://github.com/BigSimmo/Database/pull/2023 | 5bcce8927e60ce8271473823460aa281d79fca1f | src/app/api Zod row contracts (ledger #212 tranche 3) + 2 ledger inbox requests | Approved -- 4 unchecked structure-asserting casts on inbound DB/RPC data replaced with constraint-backed Zod assertions in new src/lib/validation/row-contracts.ts; 3 outbound Json telemetry casts audited and deliberately left; no src/lib/rag/** edit so ragRanking is false; 4 unrealistic document_labels test fixtures corrected without weakening assertions | verify:pr-local 13 steps green (format:changed, lint, typecheck, check:ledger-write-discipline); npm run test 6699 passed with only 2 pre-existing failures proved identical on a clean worktree at merge base d02767184; build exit 0; check:rag:fixtures 36 golden cases; api-row-contract 27/27; pr-policy evaluator 0 errors 0 warnings |

docs/outstanding-issues-inbox/2f5fbcde-438b-47ab-a1b7-01309802d935.json

@@ -0,0 +1,11 @@
+{
+  "version": 2,
+  "id": "2f5fbcde-438b-47ab-a1b7-01309802d935",
+  "createdOn": "2026-08-17",
+  "action": "update",
+  "payload": {
+    "id": "#212",
+    "detail": "Corrected 2026-08-17: the previous text was stale in two ways. (1) The \"40 unsafe casts\" headline is wrong. Measured on main at d0276718 across src/, worker/ and scripts/: 80 \"as unknown as\" and 60 \"JSON.parse\" occurrences -- but most are legitimate OUTBOUND serialization (pgvector query_embedding args, telemetry inserts cast to Json) or client handles, to which no Zod schema applies, so the remediable number is far lower than either figure suggests. (2) The named remaining population was wrong: src/app/api/documents/route.ts, src/app/api/ingestion/{batches,jobs,quality}/route.ts and src/app/api/jobs/route.ts were listed as holding 11 casts; each in fact holds ZERO \"as unknown as\" and ZERO JSON.parse. STATUS: src/lib/rag/** is COMPLETE for this class of defect as of tranche 1 (PR #1946, rag.ts, live-verified) and tranche 2 (PR #1981, rag-candidate-sources.ts); five casts remain there and all five are deliberate (outbound pgvector args plus one client handle). Tranche 3 covered src/app/api/**: an audit of all 41 route files found only 3 \"as unknown as\" (all outbound telemetry cast to Json, correctly left alone) and no JSON.parse at all, and confirmed every request body is already validated through parseJsonBody with a Zod schema. The genuine targets were a different metric -- unchecked casts that assert STRUCTURE onto inbound DB/RPC data -- of which there were 4, now replaced by src/lib/validation/row-contracts.ts: search_document_chunks RPC rows, the document_chunks table fallback rows, document_labels rows, and the search_schema_health payload (declared Returns: Json, so previously a structure claim with nothing behind it). REMAINING: worker/main.ts (11 \"as unknown as\") is the next and largest cluster, deferred as its own tranche because ingestion is a separate risk domain. Scripts are not production paths.",
+    "source": "ledger #212 tranche 3 (PR for src/app/api row contracts), session 2026-08-17"
+  }
+}

docs/outstanding-issues-inbox/aa0eb6ae-8f6a-4440-8a15-6ba0d17f812d.json

@@ -0,0 +1,14 @@
+{
+  "version": 2,
+  "id": "aa0eb6ae-8f6a-4440-8a15-6ba0d17f812d",
+  "createdOn": "2026-08-17",
+  "action": "add",
+  "payload": {
+    "pri": "P1",
+    "type": "issue",
+    "summary": "Decide whether a fabricated similarity of 1 on document-summary rows may earn the high confidence label a clinician reads",
+    "detail": "buildDocumentSummaryResults (src/lib/rag/rag-row-contracts.ts) stamps similarity: 1 on document-summary rows -- a fabricated score, not a measured cosine -- and does NOT set similarity_origin. deriveConfidence (src/lib/rag/rag-answer-support.ts:32-36) computes strongestNonSynthetic by EXCLUDING rows tagged similarity_origin === \"synthetic_text\", so an untagged fabricated 1.0 IS counted, and line 36 gates the \"high\" verdict on strongestNonSynthetic >= 0.82 with at least 2 accepted citations. A document summary therefore reaches the \"high\" confidence label on a score nobody measured. Note the mechanism precisely: buildDocumentSummaryResults does NOT set the tag -- the three call sites that DO tag synthetic scores are in rag-candidate-sources.ts (lines 626, 721, 934), and it is the ABSENCE of the tag here that admits the fabricated score. docs/clinical-hazard-analysis.md H5a records the adjacent document-lookup fast-path hazard. QUESTION: should a fabricated similarity count toward \"high\"? Adding the tag would demote these answers to medium or low. This is a clinical-governance decision before it is an engineering one: either way it needs its own design, discriminating offline tests that separate tagged from untagged rows, and a live eval-canary before/after pair per docs/rag-behaviour/. Deliberately excluded from PR #1981 and from the #212 tranche 3 PR rather than bundled, because it changes clinical output. Verified on main at d0276718.",
+    "source": "ledger #212 tranche 3 (src/app/api row contracts), session 2026-08-17",
+    "issueUlid": "01M07DCMBNJ912J9F17RHNW2AJ"
+  }
+}

src/app/api/documents/[id]/labels/route.ts

@@ -7,9 +7,10 @@ import { normalizeDocumentLabelForStorage } from "@/lib/document-tags";
 import { invalidateRagCachesForDocumentMutation } from "@/lib/rag/rag";
 import { createAdminClient } from "@/lib/supabase/admin";
 import { AuthenticationError, requireAuthenticatedUser, unauthorizedResponse } from "@/lib/supabase/auth";
-import type { DocumentLabel, DocumentLabelType } from "@/lib/types";
+import type { DocumentLabelType } from "@/lib/types";
 import { parseJsonBody } from "@/lib/validation/body";
 import { parseRouteParams } from "@/lib/validation/params";
+import { assertDocumentLabelRows } from "@/lib/validation/row-contracts";
 
 export const runtime = "nodejs";
 
@@ -101,7 +102,9 @@ async function selectLabels(supabase: ReturnType<typeof createAdminClient>, docu
     .order("label", { ascending: true });
 
   if (error) throw new Error(error.message);
-  return (data ?? []) as DocumentLabel[];
+  const rows = data ?? [];
+  assertDocumentLabelRows(rows);
+  return rows;
 }
 
 export async function POST(request: Request, { params }: { params: Promise<{ id: string }> }) {

src/app/api/documents/[id]/search/route.ts

@@ -11,22 +11,14 @@ import { AuthenticationError, unauthorizedResponse } from "@/lib/supabase/auth";
 import { enforceDocumentReadRateLimit, withOwnerReadScope } from "@/lib/public-api-access";
 import { parseRouteParams } from "@/lib/validation/params";
 import { parseRequestQuery, queryInteger } from "@/lib/validation/query";
+import {
+  assertDocumentChunkSearchRpcRows,
+  assertDocumentChunkSearchTableRows,
+  type DocumentChunkSearchRow,
+} from "@/lib/validation/row-contracts";
 
 export const runtime = "nodejs";
 
-type DocumentChunkSearchRow = {
-  id: string;
-  page_number: number | null;
-  chunk_index: number;
-  section_heading: string | null;
-  content: string;
-  image_ids: string[] | null;
-  text_rank?: number | null;
-  trigram_score?: number | null;
-  metadata?: Record<string, unknown> | null;
-  index_generation_id?: string | null;
-};
-
 const maxSearchTerms = 8;
 const defaultSearchLimit = 20;
 const maxSearchLimit = 60;
@@ -218,7 +210,9 @@ export async function GET(request: Request, { params }: { params: Promise<{ id:
     });
 
     if (!rpcError) {
-      const results = ((rpcData ?? []) as DocumentChunkSearchRow[])
+      const rpcRows = rpcData ?? [];
+      assertDocumentChunkSearchRpcRows(rpcRows);
+      const results = rpcRows
         .filter((row) =>
           isCommittedGenerationMetadata({
             rowMetadata: generationMetadataForRow(row),
@@ -262,7 +256,9 @@ export async function GET(request: Request, { params }: { params: Promise<{ id:
     if (error) throw new Error(error.message);
 
     const importantTerms = importantTermsFor(terms);
-    const committedData = ((data ?? []) as DocumentChunkSearchRow[]).filter((row) =>
+    const fallbackTableRows = data ?? [];
+    assertDocumentChunkSearchTableRows(fallbackTableRows);
+    const committedData = fallbackTableRows.filter((row) =>
       isCommittedGenerationMetadata({
         rowMetadata: generationMetadataForRow(row),
         committedGeneration,

src/app/api/setup-status/route.ts

@@ -7,6 +7,7 @@ import { createAdminClient } from "@/lib/supabase/admin";
 import { AuthenticationError, requireAuthenticatedUser } from "@/lib/supabase/auth";
 import { formatSupabaseUnavailableError, isSupabaseUnavailableError, probeSupabaseHealth } from "@/lib/supabase/health";
 import { checkSupabaseProjectConfig, formatSupabaseProjectCheck } from "@/lib/supabase/project";
+import { assertSearchSchemaHealth } from "@/lib/validation/row-contracts";
 
 export const dynamic = "force-dynamic";
 export const runtime = "nodejs";
@@ -153,13 +154,6 @@ async function readSchemaStatus(supabase: AdminClient | null) {
   }
 }
 
-type SearchSchemaHealth = {
-  ok?: boolean;
-  missing?: string[];
-  vector_extension_schema?: string | null;
-  checked_at?: string;
-};
-
 async function readSearchSchemaStatus(supabase: AdminClient | null) {
   const label = "Search RPC and vector indexes";
   if (!requiredSupabaseEnvPresent) {
@@ -176,8 +170,9 @@ async function readSearchSchemaStatus(supabase: AdminClient | null) {
     if (error) {
       return check("search", label, "needs_setup", "Search health checks are temporarily unavailable.");
     }
-    const health = (data ?? {}) as SearchSchemaHealth;
-    const missing = Array.isArray(health.missing) ? health.missing : [];
+    const health = data ?? {};
+    assertSearchSchemaHealth(health);
+    const missing = health.missing;
     if (!health.ok || missing.length > 0) {
       return check(
         "search",

src/lib/validation/row-contracts.ts

@@ -0,0 +1,218 @@
+import { z } from "zod";
+import { logger } from "@/lib/logger";
+import type { DocumentLabel, DocumentLabelType } from "@/lib/types";
+
+/**
+ * Runtime shape contracts for database rows and RPC payloads consumed by `src/app/api/**`.
+ *
+ * Route handlers had been asserting these shapes with a bare `as SomeRow[]`, which is a
+ * compile-time claim about a live database that is known to drift: `docs/outstanding-issues.md`
+ * `#316` records 20 missing indexes and 10 retrieval RPC bodies diverging from this repo's
+ * migrations, with weekly live-drift red since 2026-07-26. A renamed column or a numeric
+ * returned as a string then does not fail — it silently mis-scores, mis-filters, or mis-renders.
+ * This module is the `src/app/api` counterpart of `src/lib/rag/rag-row-contracts.ts` and
+ * deliberately duplicates that module's small validate-log-throw core rather than importing it,
+ * so that hardening an API route never edits a protected RAG ranking surface.
+ *
+ * Every contract here follows the same two rules:
+ *
+ * - **`z.looseObject`, never `z.object`.** Zod strips unknown keys by default, which would be
+ *   silent data loss whenever the live column set is ahead of this repo. Unknown keys pass
+ *   through untouched.
+ * - **Pin only what a constraint backs.** Each required field below is `not null` or carries a
+ *   `check` in `supabase/schema.sql`, cited per contract, so requiring it cannot reject a row
+ *   the database would accept today. Anything unconstrained stays permissive.
+ */
+
+/** Cap on reported issues; a wholesale shape change would otherwise report one per row. */
+const MAX_REPORTED_ISSUES = 5;
+
+/**
+ * Thrown when a database row or RPC payload does not satisfy its route's contract.
+ *
+ * The message carries only Zod issue paths and codes — never a row value. These rows carry
+ * clinical document text and owner identifiers, so echoing one into a log or an error response
+ * would leak content past the privacy boundary `query-privacy.ts` maintains.
+ */
+export class ApiRowShapeError extends Error {
+  readonly source: string;
+  readonly issues: string[];
+
+  constructor(source: string, issues: string[]) {
+    super(`"${source}" returned data that does not match its route contract: ${issues.join("; ")}`);
+    this.name = "ApiRowShapeError";
+    this.source = source;
+    this.issues = issues;
+  }
+}
+
+function describeIssues(error: z.ZodError): string[] {
+  const described = error.issues
+    .slice(0, MAX_REPORTED_ISSUES)
+    .map((issue) => `${issue.path.join(".") || "<root>"}: ${issue.message}`);
+  const remaining = error.issues.length - described.length;
+  return remaining > 0 ? [...described, `and ${remaining} more`] : described;
+}
+
+/**
+ * Shared validate-log-throw step. Kept separate so every contract fails identically, and logs
+ * before throwing so drift stays visible even where a caller catches and degrades.
+ */
+function assertAgainst(schema: z.ZodType, value: unknown, source: string): void {
+  const parsed = schema.safeParse(value);
+  if (parsed.success) return;
+  const issues = describeIssues(parsed.error);
+  logger.error("api_row_shape_mismatch", {
+    source,
+    issues,
+    rowCount: Array.isArray(value) ? value.length : null,
+  });
+  throw new ApiRowShapeError(source, issues);
+}
+
+/**
+ * Columns shared by both in-document search paths, backed by `public.document_chunks`
+ * in `supabase/schema.sql`: `chunk_index` and `content` are `not null`, `image_ids` is
+ * `not null default '{}'`, and `id` is the primary key. `page_number` and `section_heading`
+ * are nullable columns and stay `.nullable()`.
+ *
+ * The two score fields are `.nullish()` — absent or null already flows through the caller's
+ * `Number(row.text_rank ?? 0)` handling unchanged — but a *string where a number belongs* is
+ * rejected. That is the case worth catching: `scoreChunk` coerces with `Number(...)`, so a
+ * stringified rank becomes a silently different score rather than an error, and `score`
+ * orders the snippets a clinician reads.
+ */
+const documentChunkSearchBase = z.looseObject({
+  id: z.string().min(1),
+  page_number: z.number().int().nullable(),
+  chunk_index: z.number().int(),
+  section_heading: z.string().nullable(),
+  content: z.string(),
+  image_ids: z.array(z.string()).nullish(),
+  text_rank: z.number().nullish(),
+  trigram_score: z.number().nullish(),
+});
+
+/**
+ * Rows from the `search_document_chunks` RPC.
+ *
+ * The RPC's `returns table (...)` in `supabase/schema.sql` lists exactly the eight columns
+ * above — it returns neither `metadata` nor `index_generation_id`, because migration
+ * `20260717130000_filter_search_document_chunks_committed_generation.sql` moved the
+ * committed-generation filter into the SQL body itself. The route's client-side
+ * `isCommittedGenerationMetadata` pass over these rows is therefore inert (it fails open on
+ * absent generation data) and load-bearing only on the table fallback below. That is left
+ * exactly as it behaves today; this contract only stops the shape claim from being unchecked.
+ */
+const documentChunkSearchRpcRowsSchema = z.array(documentChunkSearchBase);
+
+/**
+ * Rows from the `document_chunks` table fallback used when the RPC is unavailable.
+ *
+ * This path selects `metadata` and `index_generation_id` explicitly, and here the
+ * committed-generation filter genuinely gates which chunks a clinician sees. `metadata` is
+ * `not null` on the table but is bare `jsonb` with no `jsonb_typeof` check, so it is
+ * deliberately **not** pinned to an object — the caller passes it to `committedIndexGeneration`,
+ * which already accepts `unknown`. `index_generation_id` is a nullable `uuid` column.
+ */
+const documentChunkSearchTableRowsSchema = z.array(
+  documentChunkSearchBase.extend({
+    metadata: z.unknown(),
+    index_generation_id: z.string().nullish(),
+  }),
+);
+
+export type DocumentChunkSearchRow = z.infer<typeof documentChunkSearchBase> & {
+  metadata?: unknown;
+  index_generation_id?: string | null;
+};
+
+/** Validate `search_document_chunks` RPC rows before they are scored and ordered. */
+export function assertDocumentChunkSearchRpcRows(rows: unknown): asserts rows is DocumentChunkSearchRow[] {
+  assertAgainst(documentChunkSearchRpcRowsSchema, rows, "search_document_chunks");
+}
+
+/** Validate `document_chunks` fallback rows before generation filtering, scoring, and ordering. */
+export function assertDocumentChunkSearchTableRows(rows: unknown): asserts rows is DocumentChunkSearchRow[] {
+  assertAgainst(documentChunkSearchTableRowsSchema, rows, "document_chunks.portable_ilike_fallback");
+}
+
+/**
+ * Rows from `public.document_labels`.
+ *
+ * The two enum pins are exactly the table's `check` constraints in `supabase/schema.sql`:
+ * `label_type` is checked against the same fourteen values as `DocumentLabelType` in
+ * `src/lib/types.ts`, and `source` against `('generated', 'manual')`. Both columns are plain
+ * `text not null`, so the hand-written union in `DocumentLabel` was previously an unchecked
+ * narrowing — an out-of-union `label_type` would have flowed straight into the clinical badge
+ * and filter surfaces that switch on it. `confidence` is `real not null check (0..1)`, and
+ * `label`/`document_id`/`id` are all `not null`.
+ *
+ * `metadata` is `not null` but is bare `jsonb` with no `jsonb_typeof` check, so it is left
+ * unpinned rather than asserted to be an object. `DocumentLabel` types it as
+ * `Record<string, unknown> | null`; that remains an unproven narrowing, unchanged by this
+ * contract and flagged here rather than silently blessed.
+ */
+const documentLabelRowsSchema = z.array(
+  z.looseObject({
+    id: z.string().min(1),
+    document_id: z.string().min(1),
+    owner_id: z.string().nullish(),
+    label: z.string(),
+    // `satisfies` pins this list to `DocumentLabelType`, so widening the union without
+    // widening the database `check` (or the reverse) fails the build rather than silently
+    // admitting a label type the badge surfaces cannot render.
+    label_type: z.enum([
+      "site",
+      "topic",
+      "document_type",
+      "medication",
+      "risk",
+      "setting",
+      "workflow",
+      "population",
+      "service",
+      "clinical_action",
+      "care_phase",
+      "document_intent",
+      "content_feature",
+      "custom",
+    ] satisfies [DocumentLabelType, ...DocumentLabelType[]]),
+    source: z.enum(["generated", "manual"]),
+    confidence: z.number().min(0).max(1),
+  }),
+);
+
+/** Validate `document_labels` rows before they reach clinical badge and filter surfaces. */
+export function assertDocumentLabelRows(rows: unknown): asserts rows is DocumentLabel[] {
+  assertAgainst(documentLabelRowsSchema, rows, "document_labels");
+}
+
+/**
+ * The `search_schema_health` RPC payload.
+ *
+ * This RPC is declared `Returns: Json` in `src/lib/supabase/database.types.ts` — genuinely
+ * unstructured as far as the client is concerned — so the route's previous
+ * `(data ?? {}) as SearchSchemaHealth` was a structure claim with nothing behind it. The
+ * function's `jsonb_build_object` in `supabase/schema.sql` always emits `ok`, `missing`,
+ * `vector_extension_schema` and `checked_at`, so pinning them cannot reject a healthy
+ * response. `vector_extension_schema` is a nullable local, and `checked_at` stays permissive
+ * because the route never reads it.
+ *
+ * The caller already wraps this read in a `try`/`catch` that reports `needs_setup`, so a
+ * mismatch degrades to the same status a falsy `ok` produces today — with a logged issue path
+ * instead of a silent `Missing or stale search schema items: unknown.`
+ */
+const searchSchemaHealthSchema = z.looseObject({
+  ok: z.boolean(),
+  missing: z.array(z.string()),
+  vector_extension_schema: z.string().nullish(),
+  checked_at: z.unknown(),
+});
+
+export type SearchSchemaHealth = z.infer<typeof searchSchemaHealthSchema>;
+
+/** Validate the `search_schema_health` payload before it drives the setup-status verdict. */
+export function assertSearchSchemaHealth(value: unknown): asserts value is SearchSchemaHealth {
+  assertAgainst(searchSchemaHealthSchema, value, "search_schema_health");
+}