fix(verify): bind call summary to audit evidence

[athena-kanellatou]

What

Bind the claim’s audit-derived call metadata to the supplied audit bundle during offline verification.

"verify_audit_bundle()" now re-derives the tool-call count, allowed/denied/faulted counts, and unique invoked tools from "tool_call" entries, then compares them with "trace.tool_transcript.call_count" and the five corresponding "gateway.call_summary" fields.

Why

A validly re-signed claim could previously carry call summaries that contradicted the separately supplied, correctly signed audit evidence while both signature and audit-bundle verification passed.

Closes #582.

Security impact

This strengthens audit-chain integrity by making contradictory call metadata a fatal audit-bundle verification failure, even when the modified claim has a valid signature. The verifier uses the same derivation semantics as the runtime producer:

• total: entries with "entry_type == "tool_call""
• allowed: "policy_decision == "allow""
• denied: "policy_decision in ("deny", "advisory_deny")"
• faulted: "policy_decision == "fault""
• invoked tools: sorted unique non-null "tool_name" values

Fields that cannot always be reconstructed from exported tool-call entries remain out of scope.

Test plan

• Targeted "pytest" verification tests pass ("59 passed")
• "ruff check" passes for the modified verifier and test files
• "mypy" passes
• Manual test performed

Regression coverage independently mutates and re-signs each of the six bound fields, confirms claim signature verification still passes, and confirms audit-bundle verification fails with the corresponding mismatch.

DCO sign-off

• I certify that I wrote or have the right to submit this contribution, and I agree to the Developer Certificate of Origin (https://developercertificate.org).

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[athena-kanellatou]

@Yatsuiii The implementation is ready and CI is green. I’d appreciate your review when you have a chance, as you offered on #582. Thank you!

[Yatsuiii]

Reviewed at a211ec2, as offered on #582.

The verifier's derivation matches the producer's in src/cmcp_runtime/session/manager.py:265-279 field for field: same entry_type == "tool_call" filter, same allow / ("deny", "advisory_deny") / fault predicates, same sorted() over the non-null tool_name set. The producer sorts too, so binding to sorted order will not reject a valid claim.

Ran the checkbox you left unticked: mypy src gives Success: no issues found in 64 source files. tests/unit/test_verify_command.py and tests/conformance/test_audit_conformance.py give 53 passed.

Two things I checked that are fine. Empty entries cannot reach the new block, verify.py:564 returns first. A claim with no gateway.call_summary fails closed, since .get(field) returns None against an integer.

Two findings, neither blocking.

_summary(chain) now derives the five fields with the same rule the verifier uses. That fixes the hardcoded tools_invoked=["tool.a"] that contradicted its own bundle, but the fixture can no longer disagree with the verifier, so a wrong derivation would pass on both sides. The six mutation cases pin the negative direction. The positive direction is now checked against a value the test computed itself. Literal expected values matching the fixture's chain would keep that pin.

Booleans satisfy the integer comparisons. A claim carrying "tool_calls_total": true passes against a real count of 1, because True != 1 is False. Same for false against zero. The count is not misreported so I would not hold the PR for it, but isinstance(v, int) and not isinstance(v, bool) closes it. Same class as the slsa_level boolean in trace-tests#85.

Binding looks correct and each of the six fields has its own regression.

[Yatsuiii]

Both addressed. The integer guard reads correctly, and pinning in the unit test rather than the fixture is the better call. Nothing further from me.

[athena-kanellatou]

Thanks for the thorough review and confirmation!

[imran-siddique]

Reviewed at 46dee44. The verifier derives all six reconstructable fields with the same predicates and ordering as the runtime producer, rejects bool-as-int values, and the regression suite independently re-signs each contradictory claim so signature PASS / audit bundle FAIL is causal evidence. Empty bundles are already rejected before this block; no unresolved review threads remain; all six Linux/Windows CI matrix jobs, CodeQL, and governance passed. No blocking findings.