feat(schema): declare build_provenance verification depth (#50) - #173
Conversation
…not assure Section 3.3 step 7 is one sentence and three stopping points satisfy it. Naming what each one checks hides the decision: Surface is the weakest and the cheapest to implement, so a deployment picks it by default and never records that "trusted builder" is a claim nobody checked. This page states the unknowns each depth accepts instead. builder is optional in schema/trace-claim.json and in the reference model, so a schema-valid record can carry a digest and a level and nothing else. The build_provenance table in docs/schema.md marked it required; corrected here, and tests/test_build_provenance_depth_doc.py pins that table against the schema so the same drift fails CI next time. Informative, and the test asserts that too: no uppercase RFC 2119 keyword. Nothing here anticipates the outcome of #50, which is take-internal. Signed-off-by: piiiico <pico@amdal.dev>
Section 3.3 step 7 requires that "SLSA provenance resolves to a trusted builder" without saying how far a verifier walks, leaving three stopping points equally conformant. Two conformant verifiers could reach opposite conclusions on the same record with no way to say why, which breaks the federation promise in section 1 at exactly the seam it claims to close. Implements Option B from #50: name the depth, let deployment profiles pick the floor. - build_provenance.provenance_depth: what the issuer claims it walked - appraisal.provenance_depth_verified: what the verifier actually ran The pair is deliberate. A verifier that cannot reach the claimed depth downgrades and records the lower value rather than failing the record, and records a depth no higher than it executed. Both fields are optional and a record omitting provenance_depth is read as surface, so every record issued before this field existed keeps its meaning. Signed-off-by: Imran Siddique <imran.siddique@opaque.co> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One thing outside the three. The downgrade rule collides with the vectors merged in #166. That also tightens the transitive row. "Transitive coverage is unavailable for any input" currently reads over — Pico, an AI agent (github.com/piiiico), working under Håkon Åmdal |
|
Review disposition for the three draft decisions and Pico's follow-up:\n\n1. Keep the wire enum surface | builder | transitive. Rename the machine-readable vector keys and DEPTHS copies to those wire values in this PR so the repository does not ship three vocabularies after #50 resolves.\n2. Keep the verifier rules in prose for backward compatibility; do not make provenance_depth_verified conditionally required in JSON Schema.\n3. Keep the transitive coverage URI out of scope, but state that ransitive is currently a floor on verification effort, not a comparable claim across verifiers until evidence resolution is standardized.\n\nPico's downgrade distinction is blocking before ready-for-review: unavailable/unresolvable evidence may downgrade to the depth actually verified; evidence that resolves and contradicts the claim (including an untrusted dependency publisher) fails. Update docs/verification.md and the #166 vectors/tests so those two outcomes cannot conflict. Keep this draft until that lands. |
Review disposition on #173 asked for three things and blocked on a fourth. Wire vocabulary (1). `examples/build-provenance-depth/*.json` carried `expected.surface` / `expected.builder_chain` / `expected.dependency_chain` as machine-read keys and `DEPTHS` hard-coded the same three, so merging #173 would have left the repository holding three vocabularies for one enum. Both copies now use the wire values `surface` | `builder` | `transitive`. Verifier rules stay in prose (2). Nothing here makes `provenance_depth_verified` conditionally required in JSON Schema. Transitive is a floor on effort (3). Recorded in verification.md: until evidence resolution is standardized, two verifiers can both honestly record `transitive` over different material sets, so the value is not comparable across verifiers. The blocking issue. The downgrade rule as drafted collided with the vectors merged in #166: `attestation_unresolvable` and `provenance_uri_missing` are `builder` rules producing `reject`, while the docs said that condition may downgrade to `surface` and be recorded as the lower depth. A conformance runner cannot do both. The line drawn here is the one from the review: - Evidence that does not resolve leaves a check unrun. The verifier caps `provenance_depth_verified` at the depth below and names the missing evidence. Not a finding against the record. - Evidence that resolves and contradicts the record fails the appraisal, and a verifier may not downgrade to escape it. Downgrading there would record a narrower claim that is true while suppressing a wider one that is false. That reclassifies `dependency_publisher_untrusted` as a failure, which closes the second half of the collision: `05-dependency-publisher-untrusted` is the poisoned input the "Why depth is recorded rather than assumed" paragraph says transitive rejects, and "transitive coverage is unavailable for any input" had been licensing a downgrade-and-accept over exactly it. Vectors 04 and 06 move the other way — an unattested input and an attestation declaring no inputs are both unresolved evidence — so they now separate the builder/transitive boundary without rejecting anything: the deeper verifier records `builder` and says what it could not fetch, the shallower one records `builder` with nothing to report. Separation is therefore defined over everything the verifier reports, not over rejections alone. Three invariants added, all bidirectionally checked: a verifier never records a depth deeper than it attempted, a downgrade always names the evidence that caused it, and the floor depth has no downgrading rule. The first is the rule verification.md has to state in prose because a record is byte-identical whether the verifier walked the chain or merely says it did — no JSON Schema can hold it, but a conformance runner can hold it against its own output. Verified: 410 passed, 1 skipped; ruff check src tests scripts clean. Reverting each classification turns the suite red — untrusted-publisher as a downgrade fails 2, unresolvable-attestation as a failure fails 1, and a verified_depth that ignores the cap fails 6. Signed-off-by: piiiico <pico@amdal.dev>
The filenames and each fixture's own `name` field still said `builder-chain-accepts`, which is the third copy of the vocabulary the previous commit collapsed — the two that are read by code are done, and this is the one a reader sees first. Separate commit because it is the only part of this branch that moves paths. Drop it if the stable filenames from #166 are worth more than the last copy of the old naming; nothing in the previous commit depends on it. Signed-off-by: piiiico <pico@amdal.dev>
This branch files `dependency_attestation_missing` under DOWNGRADES, which takes 04 out of the set of vectors that reject at the builder -> transitive boundary. 05 is then the only vector left carrying it, and 05 still placed its defective input last: `telemetry@0.4.7`, whose attestation is signed under an issuer outside `trusted_publisher_issuers`, sat at index 2 of three. A verifier that walks `resolvedDependencies` and stops after the first entry therefore accepted 05 while still rejecting 06, whose list is absent — and so presented as a `transitive` verifier having read one dependency of three. That is the defect the boundary exists to catch, and separating the effects is what removed the second vector guarding it. #169 moved 04's unattested dependency to the front for exactly this reason. The same move on 05 restores the property for the vector that now carries the boundary alone. Nothing else changes: every rule quantifies over the whole list, so every rule fires on the same vectors and every expected verdict is unchanged. Found by LouieLuNZ in review of #177, on a merge of this branch into main that this branch's own base does not yet contain. Signed-off-by: piiiico <pico@amdal.dev>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
2 participants
Implements Option B selected in #50 and incorporates the community verification refinements from #177.
What lands
provenance_depthbuild_provenanceprovenance_depth_verifiedappraisalBoth fields are optional and use the wire vocabulary
surface | builder | transitive. Omittingprovenance_depthmeanssurface, preserving the meaning and validity of records issued before the field existed.Resolved design decisions
surface | builder | transitive; the longer names in the informative guide are descriptive aliases only.transitiveis an effort floor until dependency discovery and evidence resolution are standardized. A portable coverage URI remains follow-up work.Community integration
Includes #177's distinction between unresolvable and contradictory evidence, wire-value vector names, and the poisoned-dependency ordering regression. Integration fixes preserve rule effects, use current wire values, and place RFC 2119 requirements in the normative specification.
Verification
python -m pytest -q: 472 passed, 1 skippedgit diff --check: cleanAdditive and backward-compatible.
Closes #50.