Skip to content

feat(migrate): versioned schema migrations for the on-disk KB - #202

Merged
plind-junior merged 1 commit into
vouchdev:testfrom
dripsmvcp:feat/200-schema-migrations
Jun 10, 2026
Merged

feat(migrate): versioned schema migrations for the on-disk KB#202
plind-junior merged 1 commit into
vouchdev:testfrom
dripsmvcp:feat/200-schema-migrations

Conversation

@dripsmvcp

Copy link
Copy Markdown
Contributor

Summary

Implements vouch migrate (#200): a versioned, reversible, audit-logged schema
migration runner so the .vouch/ contract can evolve with the pydantic models
without silently breaking KBs in the wild. Migrations are data, not code
yaml manifests in a repo-root migrations/ directory, one consecutive version
step each.

What's new

CLI (a migrate group; the legacy integer-format migration is preserved on
the no-subcommand path — see Design below):

vouch migrate status                 # current schema version, target, pending migrations
vouch migrate plan [--to 0.3.0]      # dry-run: every file each pending migration changes
vouch migrate apply [--yes]          # apply pending migrations (audit-logged, atomic)
vouch migrate rollback               # reverse the most recently applied migration
vouch migrate verify                 # parse-load every artifact under the current version

New package src/vouch/migrations/runner (status/plan/apply/rollback/
verify), manifest (parse + verbs), rewriter (atomic write + transforms),
journal (rollback journal), schema (the version stamp), semver.

On disk.vouch/schema_version (single-line semver; absent ⇒ baseline
0.1.0, so existing KBs keep loading); vouch init stamps it. Rollback journals
live at .vouch/migrations/rollback-<id>.jsonl.

Manifests — repo-root migrations/ with a format README; transform verbs
rename / default / drop / split / merge. No real manifests ship yet
(the first lands with ROADMAP 0.2's multi-dim scopes, #2); the runner is
exercised by synthetic manifests under tests/fixtures/migrations/.

Safety model

  • Atomic per file — temp + fsync + os.replace; a file is always its old
    or new bytes, never torn.
  • Reversible — the prior content of every touched file is journalled before
    any rewrite, so applyrollback restores exact bytes.
  • Crash-safe — journal written first, schema_version bumped last; an
    interrupted apply leaves the KB at its prior version with a journal to recover.
  • Precondition — refuses to run with pending proposals (won't rewrite a
    reviewer's in-flight queue).
  • state.db is disposable — the CLI rebuilds it after apply.

Design decisions (please sanity-check)

  1. migrations.pymigrations/ package. The existing integer "format"
    migration (config.yaml version / KB_FORMAT_VERSION) is moved into
    _legacy.py verbatim and re-exported, so from vouch import migrations
    and the existing test_migrations.py are unaffected.
  2. vouch migrate is now a group with invoke_without_command=True. With no
    subcommand it runs the legacy format migration exactly as before (all 5
    existing CLI tests pass unchanged); the subcommands drive the new semver flow.
    The two version axes — integer directory-format vs semver model-schema
    are intentionally distinct.
  3. Base branch = test (matches every recent PR; test is a strict superset
    of main).

Acceptance criteria

  • A KB in old form loads cleanly after a multi-step apply; every artifact
    round-trips through model_validate (verify)
  • vouch migrate plan is deterministic / byte-identical on repeat
  • applyrollback is byte-equivalent (journal replay)
  • Interrupting apply mid-run leaves the KB at the prior version; the journal
    restores it (tested via an injected mid-apply crash hook)
  • apply with pending proposals exits non-zero with an actionable message
  • The named synthetic 0099-test-rename.yaml manifest applies + rolls back,
    hitting exactly the right files

Test Plan

  • New tests: tests/test_schema_migrations.py (15 tests) + fixture
    tests/fixtures/migrations/0099-test-rename.yaml
  • Legacy tests/test_migrations.py (5 tests) still passes — migrate group
    preserves the old behavior
  • Lint: ruff check src tests — clean
  • Types: mypy srcSuccess: no issues found in 47 source files
  • Full suite: python -m pytest
478 passed, 5 deselected in 14.01s

Out of scope / deferred (flagged honestly)

  • 10k-artifact <5s perf — the plan is deterministic and CI asserts that;
    the 10k benchmark itself is not run in CI (kept fast). The design (per-kind
    single pass, no re-parsing) is built for it.
  • Bundle version embedding / refusing mismatched imports — a clean follow-up;
    not required by the acceptance criteria and kept out to avoid touching
    bundle.py import paths in this PR.
  • custom: path/to/migration.py transforms — explicitly v2 per the issue.

Closes #200

…ouchdev#200)

Give the .vouch/ contract a versioned, reversible, audit-logged upgrade path so
evolving the pydantic models never silently breaks a KB in the wild.

  vouch migrate status     current schema version, target, pending migrations
  vouch migrate plan        dry-run: every file each pending migration changes
  vouch migrate apply       apply pending migrations (audit-logged, atomic)
  vouch migrate rollback    reverse the most recently applied migration
  vouch migrate verify      parse-load every artifact under the current version

Migrations are data, not code: yaml manifests in a repo-root migrations/ dir,
one consecutive version step each (rename / default / drop / split / merge verbs).
A new .vouch/schema_version stamp (absent => baseline 0.1.0) gates load; init
writes it.

Safety:
- atomic per file (temp + fsync + os.replace) — never a torn write
- reversible: the prior content of every touched file is journalled to
  .vouch/migrations/rollback-<id>.jsonl before any rewrite, so apply -> rollback
  is byte-equivalent
- crash-safe: journal written first, schema_version bumped last, so an
  interrupted apply leaves the KB at its prior version with a journal to recover
- refuses to run with pending proposals (won't rewrite a reviewer's queue)
- state.db is disposable; the CLI rebuilds it after apply

migrations.py becomes a migrations/ package: the legacy integer "format"
migration (config.yaml version / KB_FORMAT_VERSION) is preserved verbatim under
`vouch migrate` with no subcommand, and re-exported so existing imports and tests
are unaffected. The two version axes (integer directory-format vs semver
model-schema) are intentionally distinct.

Closes vouchdev#200
@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4fd7ca80-2acd-446e-9c7f-ad94826a6f36

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@plind-junior

Copy link
Copy Markdown
Member

LGTM!

@plind-junior
plind-junior merged commit c24c51c into vouchdev:test Jun 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants