feat(migrate): versioned schema migrations for the on-disk KB - #202
Merged
Merged
Conversation
…ouchdev#200) Give the .vouch/ contract a versioned, reversible, audit-logged upgrade path so evolving the pydantic models never silently breaks a KB in the wild. vouch migrate status current schema version, target, pending migrations vouch migrate plan dry-run: every file each pending migration changes vouch migrate apply apply pending migrations (audit-logged, atomic) vouch migrate rollback reverse the most recently applied migration vouch migrate verify parse-load every artifact under the current version Migrations are data, not code: yaml manifests in a repo-root migrations/ dir, one consecutive version step each (rename / default / drop / split / merge verbs). A new .vouch/schema_version stamp (absent => baseline 0.1.0) gates load; init writes it. Safety: - atomic per file (temp + fsync + os.replace) — never a torn write - reversible: the prior content of every touched file is journalled to .vouch/migrations/rollback-<id>.jsonl before any rewrite, so apply -> rollback is byte-equivalent - crash-safe: journal written first, schema_version bumped last, so an interrupted apply leaves the KB at its prior version with a journal to recover - refuses to run with pending proposals (won't rewrite a reviewer's queue) - state.db is disposable; the CLI rebuilds it after apply migrations.py becomes a migrations/ package: the legacy integer "format" migration (config.yaml version / KB_FORMAT_VERSION) is preserved verbatim under `vouch migrate` with no subcommand, and re-exported so existing imports and tests are unaffected. The two version axes (integer directory-format vs semver model-schema) are intentionally distinct. Closes vouchdev#200
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Member
|
LGTM! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements
vouch migrate(#200): a versioned, reversible, audit-logged schemamigration runner so the
.vouch/contract can evolve with the pydantic modelswithout silently breaking KBs in the wild. Migrations are data, not code —
yaml manifests in a repo-root
migrations/directory, one consecutive versionstep each.
What's new
CLI (a
migrategroup; the legacy integer-format migration is preserved onthe no-subcommand path — see Design below):
New package
src/vouch/migrations/—runner(status/plan/apply/rollback/verify),
manifest(parse + verbs),rewriter(atomic write + transforms),journal(rollback journal),schema(the version stamp),semver.On disk —
.vouch/schema_version(single-line semver; absent ⇒ baseline0.1.0, so existing KBs keep loading);vouch initstamps it. Rollback journalslive at
.vouch/migrations/rollback-<id>.jsonl.Manifests — repo-root
migrations/with a format README; transform verbsrename/default/drop/split/merge. No real manifests ship yet(the first lands with ROADMAP 0.2's multi-dim scopes, #2); the runner is
exercised by synthetic manifests under
tests/fixtures/migrations/.Safety model
fsync+os.replace; a file is always its oldor new bytes, never torn.
any rewrite, so
apply→rollbackrestores exact bytes.schema_versionbumped last; aninterrupted apply leaves the KB at its prior version with a journal to recover.
reviewer's in-flight queue).
state.dbis disposable — the CLI rebuilds it after apply.Design decisions (please sanity-check)
migrations.py→migrations/package. The existing integer "format"migration (
config.yamlversion/KB_FORMAT_VERSION) is moved into_legacy.pyverbatim and re-exported, sofrom vouch import migrationsand the existing
test_migrations.pyare unaffected.vouch migrateis now a group withinvoke_without_command=True. With nosubcommand it runs the legacy format migration exactly as before (all 5
existing CLI tests pass unchanged); the subcommands drive the new semver flow.
The two version axes — integer directory-format vs semver model-schema —
are intentionally distinct.
test(matches every recent PR;testis a strict supersetof
main).Acceptance criteria
apply; every artifactround-trips through
model_validate(verify)vouch migrate planis deterministic / byte-identical on repeatapply→rollbackis byte-equivalent (journal replay)applymid-run leaves the KB at the prior version; the journalrestores it (tested via an injected mid-apply crash hook)
applywith pending proposals exits non-zero with an actionable message0099-test-rename.yamlmanifest applies + rolls back,hitting exactly the right files
Test Plan
tests/test_schema_migrations.py(15 tests) + fixturetests/fixtures/migrations/0099-test-rename.yamltests/test_migrations.py(5 tests) still passes —migrategrouppreserves the old behavior
ruff check src tests— cleanmypy src—Success: no issues found in 47 source filespython -m pytestOut of scope / deferred (flagged honestly)
the 10k benchmark itself is not run in CI (kept fast). The design (per-kind
single pass, no re-parsing) is built for it.
not required by the acceptance criteria and kept out to avoid touching
bundle.pyimport paths in this PR.custom: path/to/migration.pytransforms — explicitly v2 per the issue.Closes #200