You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Claim model has no min-evidence validator
(src/vouch/models.py:184-187), so a claim with evidence: [] is
schema-valid. As a result, the README and CONTRIBUTING contract —
Claims must cite sources. A claim without at least one Source /
Evidence id is a validation error, not a warning.
is enforced in exactly one place: proposals.propose_claim
(src/vouch/proposals.py:89-90). Every other write path to claims/<id>.yaml accepts uncited claims silently:
Bundle import — bundle._validate_content defers to Claim.model_validate (src/vouch/bundle.py:43,222-228), which
accepts evidence: []. A hand-crafted bundle with one such claim
passes import_check (ok=True, no issues) and import_apply
writes it to disk under the legitimate bundle_id. The audit
log records a clean bundle.import event for a KB that now
contains an uncited claim.
store.put_claim direct — src/vouch/storage.py:283-291
iterates claim.evidence to verify each cited id exists, but an
empty list iterates zero times. Lifecycle ops
(supersede, contradict, archive, confirm) and any caller
that constructs a Claim directly can land an uncited one.
store.update_claim — src/vouch/storage.py:308-… writes
the YAML without any citation check. A previously-cited claim
can be mutated to evidence=[] and the change persists.
This is exactly the failure mode CONTRIBUTING calls out under
"Things we won't merge":
Validation relaxations that let claims land without citations.
A claim without evidence is a working note at best — register
a source.
— except the enforcement was never on the model in the first
place, so every write path except the one explicit gate already
qualifies as the relaxed shape the team won't accept.
What you expected
Constructing a Claim with evidence=[] raises pydantic.ValidationError.
That closes all three bypass paths in one stroke:
bundle.import_apply raises (via _validate_content) before
writing.
store.put_claim(Claim(evidence=[])) raises at the Claim(...)
call.
store.update_claim(claim_with_emptied_evidence) raises the
moment a caller mutates claim.evidence = [] and tries to pass
it back through Claim(...) or pydantic re-validation.
The duplicate check inside propose_claim
(if not evidence: raise ProposalError(...)) can stay as the
nicer user-facing error message, but the load-bearing invariant
moves to the model where it belongs.
Reproduction
$ python uncited_repro.py
work dir: /tmp/vouch-uncited-XXXXXXXX
Bypass 1 (put_claim): SUCCEEDED — uncited claim on disk
YAML on disk has 'evidence: []' → True
Bypass 2 (update_claim): SUCCEEDED — citations stripped to []
bundle.import_check(uncited): ok=True issues=[]
bundle.import_apply: wrote 1 files
landed claim text: 'shipped via bundle, no citations'
landed claim evidence: []
BYPASS 3 CONFIRMED: bundle import landed an uncited claim — the
README contract was broken.
uncited_repro.py exercises all three paths against a fresh KB:
store.put_claim(Claim(id="direct-uncited", text="…", evidence=[]))
— writes claims/direct-uncited.yaml with evidence: [].
Create a cited claim, then c.evidence = []; store.update_claim(c)
— the empty list persists.
Build a bundle whose manifest is internally consistent and whose claims/bundle-uncited.yaml member has evidence: []. Call import_check (returns ok=True) and import_apply (lands the
uncited claim). Read it back via store.get_claim — the
stored claim has evidence == [].
Host: any (CLI / MCP / JSONL — every transport eventually calls
one of the three uncited code paths)
.vouch/ state
$ vouch doctor
Not relevant — the bug reproduces on a freshly-initialised KB.
Anything else
Suggested fix
src/vouch/models.py, on the Claim model:
classClaim(BaseModel):
...
evidence: list[str] =Field(
default_factory=list,
description="Source ids OR Evidence ids — both are valid citations",
)
@field_validator("evidence")@classmethoddef_at_least_one_citation(cls, v: list[str]) ->list[str]:
ifnotv:
raiseValueError(
"claim must cite at least one Source or Evidence id ""(README §'Object model'; CONTRIBUTING §'Things we won't merge')"
)
returnv
That's the entire fix on the source side. The existing guard in proposals.propose_claim becomes redundant but can stay as a
nicer error string for the CLI/JSONL front door.
Tests to add in tests/test_storage.py (round-trip per CONTRIBUTING):
test_put_claim_rejects_empty_evidence — direct store.put_claim raises.
test_update_claim_rejects_empty_evidence — mutating an
existing claim to evidence=[] raises before disk write.
In tests/test_bundle.py: test_import_rejects_uncited_claim
— bundle with a manifest-consistent claim that has evidence: [] is rejected by import_check with a clean schema validation failed: issue, and import_apply raises.
Why it's worth fixing
This is the chore(deps): bump actions/checkout from 4 to 6 #2 load-bearing guarantee vouch sells (README
§"Why this exists" point 3: "Claims must cite sources").
Today it's only enforced at one of four write paths, so every
KB on disk could already contain uncited claims from any of the
three bypassed routes.
CONTRIBUTING explicitly lists "Validation relaxations that let
claims land without citations" as a "won't merge" category —
the team has stated this is load-bearing. The fix is moving the
enforcement from one function to the model so the contract
matches the implementation everywhere.
Single-line fix on the model closes three independent
bypass paths simultaneously. Test coverage scales naturally.
What happened
The
Claimmodel has no min-evidence validator(
src/vouch/models.py:184-187), so a claim withevidence: []isschema-valid. As a result, the README and CONTRIBUTING contract —
is enforced in exactly one place:
proposals.propose_claim(
src/vouch/proposals.py:89-90). Every other write path toclaims/<id>.yamlaccepts uncited claims silently:Bundle import —
bundle._validate_contentdefers toClaim.model_validate(src/vouch/bundle.py:43,222-228), whichaccepts
evidence: []. A hand-crafted bundle with one such claimpasses
import_check(ok=True, no issues) andimport_applywrites it to disk under the legitimate
bundle_id. The auditlog records a clean
bundle.importevent for a KB that nowcontains an uncited claim.
store.put_claimdirect —src/vouch/storage.py:283-291iterates
claim.evidenceto verify each cited id exists, but anempty list iterates zero times. Lifecycle ops
(
supersede,contradict,archive,confirm) and any callerthat constructs a
Claimdirectly can land an uncited one.store.update_claim—src/vouch/storage.py:308-…writesthe YAML without any citation check. A previously-cited claim
can be mutated to
evidence=[]and the change persists.This is exactly the failure mode CONTRIBUTING calls out under
"Things we won't merge":
— except the enforcement was never on the model in the first
place, so every write path except the one explicit gate already
qualifies as the relaxed shape the team won't accept.
What you expected
Constructing a
Claimwithevidence=[]raisespydantic.ValidationError.That closes all three bypass paths in one stroke:
bundle.import_applyraises (via_validate_content) beforewriting.
store.put_claim(Claim(evidence=[]))raises at theClaim(...)call.
store.update_claim(claim_with_emptied_evidence)raises themoment a caller mutates
claim.evidence = []and tries to passit back through
Claim(...)or pydantic re-validation.The duplicate check inside
propose_claim(
if not evidence: raise ProposalError(...)) can stay as thenicer user-facing error message, but the load-bearing invariant
moves to the model where it belongs.
Reproduction
uncited_repro.pyexercises all three paths against a fresh KB:store.put_claim(Claim(id="direct-uncited", text="…", evidence=[]))— writes
claims/direct-uncited.yamlwithevidence: [].c.evidence = []; store.update_claim(c)— the empty list persists.
claims/bundle-uncited.yamlmember hasevidence: []. Callimport_check(returnsok=True) andimport_apply(lands theuncited claim). Read it back via
store.get_claim— thestored claim has
evidence == [].Environment
evidence: list[str] = Field(default_factory=list, ...)was the model shape — thevalidator was never there)
Python 3.12.13one of the three uncited code paths)
.vouch/stateNot relevant — the bug reproduces on a freshly-initialised KB.
Anything else
Suggested fix
src/vouch/models.py, on theClaimmodel:That's the entire fix on the source side. The existing guard in
proposals.propose_claimbecomes redundant but can stay as anicer error string for the CLI/JSONL front door.
Tests to add in
tests/test_storage.py(round-trip per CONTRIBUTING):test_claim_model_rejects_empty_evidence—Claim(evidence=[])raises
pydantic.ValidationError.test_put_claim_rejects_empty_evidence— directstore.put_claimraises.test_update_claim_rejects_empty_evidence— mutating anexisting claim to
evidence=[]raises before disk write.tests/test_bundle.py:test_import_rejects_uncited_claim— bundle with a manifest-consistent claim that has
evidence: []is rejected byimport_checkwith a cleanschema validation failed:issue, andimport_applyraises.Why it's worth fixing
§"Why this exists" point 3: "Claims must cite sources").
Today it's only enforced at one of four write paths, so every
KB on disk could already contain uncited claims from any of the
three bypassed routes.
claims land without citations" as a "won't merge" category —
the team has stated this is load-bearing. The fix is moving the
enforcement from one function to the model so the contract
matches the implementation everywhere.
bypass paths simultaneously. Test coverage scales naturally.
investing in: fix: validate bundle content against Pydantic models before import (#13) #34 added pydantic validation on import (so the
schema guards integrity); bug: import_check / import_apply never verify member sha256 against manifest — bundle integrity gate is bypassable #74 / PR fix(bundle): verify per-member sha256 against manifest on import #75 added per-member sha256
verification. This fix tightens the pydantic guard so its
invariants are correct. Same code area, same defence-in-depth
pattern.
Checked for duplicates
vouchdev/vouchforclaim evidence,citation required,min_length evidence,Claim validator,uncited. Closest matches: bug: propose_claim swallows I/O and parse errors in evidence validation — misleading ProposalError raised #48 (propose_claimswallows I/O errors in evidence validation — different concern;
has PR fix: narrow evidence validation to ArtifactNotFoundError in propose_claim #50), Bundle import writes unvalidated content to committed artifact directories #13 (bundle pydantic validation — closed by PR fix: validate bundle content against Pydantic models before import (#13) #34;
the schema validation it added is exactly what this fix tightens).
to the model. PR fix: validate bundle content against Pydantic models before import (#13) #34 added the model-validation pass; this fix
closes the gap left by that pass (the model itself didn't
enforce the invariant).