fix(provenance): apply the structural rules on the consumer side too - #146
Conversation
Four rules were enforced only in build_record: a tee-attested record needs attestation, an endpoint identity needs spki_sha256, an artifact identity needs digest, and issued_at must be a usable timestamp. A record does not have to come from build_record, so an attacker simply never runs them. Reported in #142 with a reproduction: a hand-assembled record claiming kind="tee-attested" with attestation: null, an endpoint identity that is a bare URL, and no issued_at, signed by a key the consumer already trusts, passed both steps of section 5. The rules move into one _check_structure called by build_record and verify_record, rather than being copied into the verifier, because two copies drift and the copy that matters is the consumer's. Closes #142 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks, pulling the structural rules onto the verify path via one _check_structure Two smaller things from the bottom of #142 are in the same function and untouched by
Both are minor next to the main fix; flagging only because they're the same few lines |
bool is an int subclass, so call_count: true passed the non-negative integer check and reported as a call count of one, false as zero. A record that says true there is malformed, and a conformance suite that silently reads it as a number is answering a question nobody asked. Found by writing the TR-TXN-002 guard test, so the test and the fix land together. Same class as the issued_at case in agentrust-io/trace-spec#146. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* measurement: does the suite notice when a conformance check breaks Fork-only. Nothing under the existing tree is modified; this adds one directory. For every Finding(..., Status.FAIL, ...) site in the conformance modules, rewrite it so the check can never fail, run the suite, count how many tests notice, restore. 23 of 33 failure paths are guarded. Ten are not, including all three of TR-TXN-001, which is the only enforcement of the Level 2 tool-transcript requirement since tool_transcript is absent from the schema's required array. None of the checks is wrong. This measures whether a regression would be caught. Signed-off-by: lywinged <48041247+lywinged@users.noreply.github.com> * test: guard the ten conformance failure paths the measurement found unverified The harness in measurement/ reported ten of thirty-three Finding(..., FAIL, ...) sites at margin zero: rewriting any of them so it could never fail left the whole suite green. Landing the instrument without the tests would have published a gap list against our own conformance suite and fixed nothing. tr_txn and tr_sca had no unit tests at all, which is why every one of their failure paths measured zero. TR-TXN-001 carried the most weight: it is the only place the Level 2 tool-transcript requirement is enforced anywhere in the suite. New: tests/unit/test_tr_txn.py, tests/unit/test_tr_sca.py. Extended: test_tr_sig.py with the two check_cmcp_runtime key-shape paths, the unsupported kty path, and the signature-present-but-uncheckable path; test_tr_anc.py with a URI that raises during parsing rather than parsing to a wrong scheme. The two TR-SIG-002 vectors are deliberately separable: an implementation that checks kty and crv without checking x passes one and fails the other, which is what makes them two vectors rather than one written twice. Re-measured: 18 of 18 checks and 33 of 33 sites verified, none unguarded. Eight checks sit at margin 1, which the report now names as the number to watch. REPORT.md and README.md updated to describe the state they measured rather than the state they found. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tr_txn): reject a boolean call_count bool is an int subclass, so call_count: true passed the non-negative integer check and reported as a call count of one, false as zero. A record that says true there is malformed, and a conformance suite that silently reads it as a number is answering a question nobody asked. Found by writing the TR-TXN-002 guard test, so the test and the fix land together. Same class as the issued_at case in agentrust-io/trace-spec#146. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Signed-off-by: lywinged <48041247+lywinged@users.noreply.github.com> Co-authored-by: lywinged <48041247+lywinged@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
verify_record compared the record's cnf.jwk against the trusted key with dict
equality. A JWK is identified by the key material; kid, use and alg are optional
members that carry none of it. key_to_jwk emits the bare {crv, kty, x}, while a
key resolved from a JWKS endpoint normally carries kid, because that is how JWKS
distinguishes keys across rotation. The two are the same public key and not the
same dict, so a consumer holding its trusted key from a JWKS rejected every
record, with a message accusing the publisher of substituting a key:
ProvenanceError: the record's embedded key is not the trusted key. A record
signed by some other key is a record about a server somebody else is
describing.
It was signed by exactly the right key. The signature verified; only the
comparison failed.
Compared by RFC 7638 thumbprint instead, which sign.jwk_thumbprint already
implements and whose docstring already states the property needed here: stable
across JWKs that differ in optional members such as kid, alg or use. A different
key still fails, and a cnf.jwk with no usable kty is now a refusal rather than an
uncaught ValueError.
This is the same comparison agentrust-io#157 has since landed in sign.verify_record, and it
is written the same way, compare_digest over the two thumbprints, so that the two
verifiers answer "is this the trusted key" identically. The error type differs
because each module raises its own: ProvenanceError here, ValueError there. After
agentrust-io#157 the divergence is visible in one tree, one verifier identifying a key by its
material and the other by its dict shape.
The regexes anchored with $, which in Python also matches before one final
newline, so publisher "did:web:acme.example\n" and a tool_catalog.hash with a
trailing newline passed verify_record. Both now anchor with \Z.
Raised on agentrust-io#146 and not part of that fix; agentrust-io#142 is closed, so both were untracked.
Six tests, each load-bearing: reverting the thumbprint comparison fails three,
reverting the anchors fails two. The embedded-key case is signed by hand, since
sign_record builds cnf itself and this implementation cannot emit a cnf.jwk
carrying kid - another implementation can, and the format permits it.
Rebased onto dd1b6e6. 345 tests pass; ruff and mypy clean.
Signed-off-by: Louielunz <48041247+lywinged@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #142.
Four structural rules lived only in
build_record:tee-attestedrecord must carry attestation evidenceendpointidentity must carryspki_sha256artifactidentity must carrydigestissued_atmust be a usable timestampA record does not have to come from
build_record. Anyone can write the JSON and sign it, so a rule on the producer side is a rule an attacker never runs. @lywinged's reproduction on #142 shows the consequence: a record claimingkind: "tee-attested"withattestation: null, an endpoint identity that is a bare URL, and noissued_at, signed by a key the consumer already trusts, passes both steps of §5. The consumer is handed the top trust tier with nothing behind it.The rules move into a single
_check_structurethat bothbuild_recordandverify_recordcall. Copying them into the verifier would have worked today and drifted later, and the copy that matters is the consumer's.This is the same principle #144 made explicit in the C2PA assertion next door: a check a verifier must not be able to skip is required, not offered. @lywinged drew that connection on the issue.
Tests. Nine cases, one per rule plus the reported forgery verbatim and a well-formed control. Run against the currently published
agentrust-trace, eight of the nine fail and the control passes, so they are guarding the actual defect.Compatibility. This rejects records the verifier previously accepted. Every such record is one
build_recordwould have refused to emit, so anything produced through the SDK is unaffected.🤖 Generated with Claude Code