Skip to content

chore(db): add schema-vs-migrations drift guard - #2674

Merged
JSONbored merged 1 commit into
mainfrom
chore/schema-migrations-drift-guard
Jul 3, 2026
Merged

chore(db): add schema-vs-migrations drift guard#2674
JSONbored merged 1 commit into
mainfrom
chore/schema-migrations-drift-guard

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • src/db/schema.ts (Drizzle ORM sqliteTable declarations) is a single shared file where two independently-valid PRs can each add a column to the same table. The existing migration column-collision check (detectColumnCollisions in src/db/migration-column-extraction.ts, added under feat(gate): detect same-table/same-column collisions across differently-numbered migrations #2551 / shipped in feat(gate): detect same-table/same-column collisions across differently-numbered migrations #2607's scripts/check-migrations.mjs) only catches two different migration files adding the same (table, column) pair — it never reads src/db/schema.ts at all (confirmed via grep: zero references). It cannot see schema.ts's declared shape silently drifting from what migrations/ actually produces when replayed against a fresh DB.
  • Adds scripts/check-schema-drift.mjs: replays every migrations/*.sql file into a fresh in-memory node:sqlite DB (mirrors test/helpers/d1.ts's TestD1Database concatenate-sorted-files-then-exec approach), introspects each table's real columns via PRAGMA table_info, imports src/db/schema.ts's exported tables and extracts their declared columns via drizzle-orm's getTableColumns, and diffs the two column-name sets per table.
  • A RAW_SQL_ONLY_TABLES allowlist (20 entries, e.g. review_audit, system_flags, orb_signals, tunables_overrides) covers the feature/aggregate tables that intentionally exist only in migrations/ and are accessed via raw SQL (env.DB.prepare(...)) rather than a Drizzle declaration — the documented house pattern ("core tables use Drizzle; feature/aggregate tables use raw-SQL migrations"). Each entry was confirmed by direct inspection to be actively read/written via raw SQL elsewhere in src/.
  • Wired into a new db:schema-drift:check npm script, added to the test:ci chain immediately after db:migrations:check, and added as a new CI step in .github/workflows/ci.yml right after "Check migrations" (gated by the same backend path filter). CI runs each check as a discrete named step rather than invoking npm run test:ci as a whole, so a workflow step was required for this to actually execute on PRs — verified npm run test:ci is never invoked from any .github/workflows/*.yml.
  • The most important test is the regression guard that runs the real diff against the repo's actual src/db/schema.ts + migrations/ and asserts zero mismatches — proving they are not already drifted today (they aren't; the check passes cleanly against the current 59 Drizzle tables + 20 allowlisted raw-SQL tables = 79 total migrated tables).

Closes #2565

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; scripts/** is excluded from Codecov's include, so this change owes no patch coverage, but test/unit/check-schema-drift-script.test.ts still covers every branch (column missing from migrations, column missing from schema.ts, table missing from migrations, undeclared migration-only table both flagged and allowlisted, non-table export skip, clean pass, CLI exit code + stderr text, and the real-repo regression guard).
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate — 0 vulnerabilities
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries
  • Full npm run test:ci (all steps, unsharded) — green: 354 test files / 6746 tests passed, 6 pre-existing skips

If any required check was skipped, explain why:

  • None skipped.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (N/A — this PR touches no auth/session/CORS surface.)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (N/A — no API/OpenAPI/MCP surface changed.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (N/A — no UI changed.)
  • Visible UI changes include a UI Evidence section below with screenshots. (N/A — this is a CI/dev-tooling-only change with no UI surface.)
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. (No changelog edit.)

UI Evidence

N/A — no UI/frontend/docs-visible change.

Notes

  • Out of scope, deliberately: updating .claude/skills/contributing-to-gittensory/reference.md's CI-checks table. That file is not in wantedPaths and is local maintainer tooling documentation, not part of this issue's described deliverables.

src/db/schema.ts is a single shared file where two independently-valid
PRs can each add a column to the same table. The existing migration
column-collision check (detectColumnCollisions, #2551) only catches two
DIFFERENT migration files adding the same column -- it never reads
schema.ts, so schema.ts's declared shape can silently drift from what
migrations/ actually produces when replayed against a fresh DB, with no
CI signal.

Adds scripts/check-schema-drift.mjs: replays migrations/*.sql into a
fresh in-memory node:sqlite DB (mirroring test/helpers/d1.ts's
TestD1Database), introspects each table's real columns via PRAGMA
table_info, and diffs that against schema.ts's declared columns via
drizzle-orm's getTableColumns. A RAW_SQL_ONLY_TABLES allowlist covers
the 20 feature/aggregate tables that intentionally exist only in
migrations/ (accessed via raw SQL, per the documented house pattern).

Wired into db:schema-drift:check, the test:ci chain next to
db:migrations:check, and a new CI step alongside "Check migrations"
(CI runs discrete steps rather than test:ci as a whole, so a step was
required for this to actually run on PRs).

Closes #2565
@dosubot dosubot Bot added the size:L label Jul 3, 2026
@codecov

codecov Bot commented Jul 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.10%. Comparing base (642b12d) to head (7fd2444).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2674   +/-   ##
=======================================
  Coverage   96.10%   96.10%           
=======================================
  Files         237      237           
  Lines       26538    26538           
  Branches     9624     9624           
=======================================
  Hits        25505    25505           
  Misses        424      424           
  Partials      609      609           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Jul 3, 2026
@JSONbored
JSONbored merged commit 02b901a into main Jul 3, 2026
11 checks passed
@JSONbored
JSONbored deleted the chore/schema-migrations-drift-guard branch July 3, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore(db): investigate a schema-vs-migrations drift guard

1 participant