AgentBoundary v0.1 conformance evaluation of AGT — pre-publication review

[sunilp]

Hi @imran-siddique and AGT team —

Following up on the conversation in #302 about PolicyDecision schema interop and drop-in compatibility across backends (APS, AGT, YAML), I wanted to share a piece I've been building in parallel and give the AGT team a 7-day right-to-respond window before publication.

I run JamJet Labs and have been authoring an open spec for AI-action receipts called AgentBoundary (jamjet-labs/agentboundary, v0.1 stable + v0.2-alpha draft). Where the PolicyDecision interop work focuses on the decision surface, AgentBoundary focuses on the downstream receipt surface — a portable, tamper-evident JSON record of what the action turned out to be, that a third party can verify without trusting the runtime.

I built a 40-scenario conformance suite and graded it against four prominent agent-governance products including AGT. AGT scored highest of the four.

What I did:

• Read docs/specs/AUDIT-COMPLIANCE-1.0.md and docs/ARCHITECTURE.md
• Built an adapter at adapters/microsoft-agt/ that translates AGT AuditEntry (+ optional DecisionBOM + workflow approval event) into an AgentBoundary v0.2-alpha receipt
• Ran all 40 conformance scenarios against adapter-translated receipts
• Per-scenario verdicts in results.md; field-by-field mapping in mapping.md

Headline:

PASS         17
PARTIAL       5
DOCS-ONLY     1
NOT COVERED  15
N/A           2
──────────────
TOTAL        40

The 15 NOT COVERED rows reflect AGT-side schema gaps where the AuditEntry doesn't carry data AgentBoundary requires:

• No normative arguments_hash (mutation defense)
• No approver identity in the audit row (approval-chain verification)
• No policy version field (downgrade defense)
• Single timestamp per entry (no issued-vs-completed split)
• No environment field (prod/staging/dev distinction)

Each maps to a known design choice on AGT's side — not bugs, but deliberate scoping. The framing in my report is that AGT and AgentBoundary's design centres are complementary: runtime enforcement + decision-lineage reconstruction (AGT) vs portable third-party verification of receipts (AgentBoundary). Two different layers; same compliance picture.

Two things AGT does better than AgentBoundary v0.2-alpha — both v0.3 adoption candidates on my side:

1. Merkle chain across actions (previous_hash). v0.2-alpha is singly-linked (prior_receipt); weaker against arbitrary-entry-reordering attacks. AGT's approach is structurally stronger.
2. DecisionBOM.completeness_score with per-BOMField reconstruction confidence. v0.2-alpha has a coarser three-tier provenance enum (observed / inferred / synthesized). A numeric confidence per field is meaningfully richer.

Both feed back into the PolicyDecision interop work cleanly — if AGT entries can map round-trip to AgentBoundary receipts (and vice versa), the receipt format becomes another axis of the drop-in replacement story #302 was about.

The ask: if any per-scenario mapping or factual claim is wrong, corrections are welcome via this issue or via PR to jamjet-labs/agentboundary within 7 days of this post. After that, the report publishes with the data as currently mapped.

Happy to discuss here or directly. The draft report is private until publication; happy to share §7.4 (the AGT section, ~600 words) for a sneak look if either of you wants one.

Thanks for shipping AGT — it raised the bar for everyone in this space.

— Sunil

[github-actions]

Welcome to the Agent Governance Toolkit! Thanks for opening your first issue.
A maintainer will review this shortly. Check our Contributing Guide.
For security issues, use private vulnerability reporting.

[github-actions]

🔴 Contributor Check: HIGH

Check	Result
Profile	HIGH
Credential	NONE
Overall	HIGH

Automated check by AGT Contributor Check.

[github-actions]

🤖 AI Agent: contributor-guide

Hi Sunil, welcome, and thank you for the detailed analysis and kind words about AGT! Your work with AgentBoundary is fascinating and aligns well with AGT's goals. This issue seems most relevant to the AuditEntry schema and DecisionBOM components. For contributing or collaborating, please check out our CONTRIBUTING.md. We'll review your findings and respond within your timeline!

[imran-siddique]

Hi @sunilp - thanks for the thorough conformance evaluation. This is exactly the kind of rigorous external validation that strengthens both projects.

On the 15 NOT COVERED gaps: You are correct that these are deliberate scoping decisions, not bugs. That said, several of the gaps you identified are worth closing:

• arguments_hash (mutation defense): agreed this belongs in AuditEntry. We have hash infrastructure already via the Merkle chain; extending it to cover action arguments is a natural next step.
• approver identity: currently tracked in the workflow approval subsystem but not surfaced in the audit row itself. Valid gap.
• policy version field: important for downgrade defense. We track policy metadata but don't embed the version in each decision record. Good catch.
• issued-vs-completed timestamps: single timestamp was a simplicity choice. Dual timestamps are worth adding for latency analysis and ordering guarantees.
• environment field: fair point, prod/staging/dev distinction should be first-class in the audit schema.

We appreciate you noting where AGT's approach is structurally stronger (Merkle chain, completeness scoring). That kind of balanced analysis builds trust.

The invitation: Rather than just responding to the report, we'd welcome you as a contributor. If you're interested, consider opening PRs for one or more of these schema extensions. The audit schema lives in \docs/specs/AUDIT-COMPLIANCE-1.0.md\ and the implementation in the Python SDK's audit module. Our CONTRIBUTING.md has the setup guide.

Your AgentBoundary receipt spec and AGT's runtime enforcement are complementary layers, as you noted. Having you contribute directly would help ensure round-trip fidelity between both formats.

No factual corrections needed on the mapping from our side. Publish with confidence.

[sunilp]

Acknowledged. Will open a first PR covering arguments_hash, approver identity, and policy version — the three integrity / non-repudiation fields, all of which extend the existing Merkle chain rather than adding new infrastructure. Dual timestamps and environment will follow in a second PR.

Publishing the conformance report this week; will link the PRs from the report once they're up.

[sunilp]

Follow-up PR for the dual-timestamp gap from this issue is now open: #2532.

Scope: adds optional issued_at and completed_at to AuditEntry, matching the additive-fields pattern from #2473. The existing timestamp is unchanged. Both new fields default to None, sit outside the v1.0 canonical hash (with the same v1.1 deferral noted in §4.3.1), and surface in the CloudEvent envelope only when set.

The third item I originally flagged — environment — was already addressed by #2527, so no separate PR is needed there.

That closes out the AuditEntry-side schema gaps from the AgentBoundary v0.1 conformance evaluation. The five DOCS-ONLY / PARTIAL rows in the published report will be re-graded against the post-#2532 schema once it lands.

[miyannishar]

Hey @sunilp — just catching up on this thread.

First off, really solid work on the conformance suite. The fact that you built adapters for four different governance products and ran all 40 scenarios tells us a lot about where the ecosystem gaps actually are versus where people assume they are. The per-field mapping tables in your repo are especially useful.

Looks like @imran-siddique already gave a detailed response on the 15 NOT COVERED gaps, and you've followed up with PR #2532 for the dual-timestamp extension. The arguments_hash and approver identity items are on our radar — both fit naturally into the Merkle chain infrastructure we already have.

Two things I want to call out from your report that we're genuinely learning from:

1. The Merkle chain vs singly-linked comparison was a good validation that our previous_hash approach is structurally stronger against reordering attacks. We hadn't benchmarked that against other receipt formats before.
2. The completeness_score vs three-tier provenance comparison is interesting — we designed the numeric confidence per BOMField for a different reason (probabilistic reconstruction from partial audit trails), but it's good to know it maps to a richer signal than coarse tiers.

I think we've met the 7-day response window here, and the outstanding schema items are tracked in their own PRs. Closing this one out — feel free to link the published report back here when it's live.

Thanks again for the rigorous analysis.