Skip to content

test(verifier-compatibility): close two margins, and record why the other two cannot - #7

Merged
lywinged merged 1 commit into
mainfrom
work/116-version-obligations
Aug 27, 2026
Merged

lywinged merged 1 commit into
mainfrom
work/116-version-obligations

Conversation

@lywinged

Copy link
Copy Markdown
Owner

Continues agentrust-io#116. The ask there was obligations 2 and 3 back "with the separation figure measured and pinned the way you did here", and "a set that separates 3 of 8 and says so is worth more than one that separates 8 of 8 and has never been asked."

Obligations 2 and 3 are already implemented: verify_record takes accepted_profiles and refuses outside it, and VerificationStatement carries profile and accepted_profiles. What was outstanding is the vector set and the number.

The module's own headline number had drifted

test_verifier_compatibility_separation.py exists so the set's separating power cannot change unnoticed. Its opening paragraph said the set separates 3 of its 8 vectors while the figure pinned in code twenty lines below said 4 of 9. A reader arrives at the docstring first and left with the superseded number.

This is the defect this module was written to prevent, happening to the module's own headline. The figure now lives in one place, next to SEPARATING, and the paragraph says so instead of restating it.

Two of four recorded margin shortfalls closed

KNOWN_THIN recorded four refusal rules carried by one vector each, the shape agentrust-io#124 established as insufficient.

10-superseded-first-in-set-innocent-record-refused closes superseded_profile_in_accepted_set. It moves two things at once: the v0.1 identifier sits first in the accepted set rather than last, so an implementation scanning only the tail passes 08 and fails here; and the record presented is an ordinary v0.2 record with nothing wrong with it. The second half is what makes it separate where 08 cannot, since 08 carries a v0.1 record that the schema refuses on its own whether or not any profile rule ran.

11-empty-profile-string-refused closes profile_absent with a claim that is present and empty. 07 removes the member outright, so an implementation testing "eat_profile" not in record passes 07 and reads this one as an unrecognised profile, or as absent and therefore current. It does not separate, and the reason is structural rather than a flaw: the schema pins eat_profile with a const, so any record that reaches this rule is already schema-invalid and a verifier implementing nothing refuses it too.

before after
vectors 9 11
separates 4 5
boundaries carried by one vector 4 2

The other two were measured and are not closable

Recorded in KNOWN_THIN with the reason, because "measured and shown to have no second axis" is a different state from "not yet done":

no_accepted_profiles fires on the verifier's own configuration before any record is read, and that configuration has one shape: the set is empty. The one other axis, pairing the empty set with a second defect the verifier would catch later, needs check_freshness, and every vector in this set asserts that is False for a good reason already written into the fixtures test: a fixed iat would make the set expire.

superseded_profile_refused needs a record carrying v0.1 and an accepted set excluding it. The set can hold only v0.2, because any other member trips unschemaed_profile_in_accepted_set first, so there is no second configuration to present.

What remains for both is varying record content, which pins nothing. No plausible implementation branches on record content when deciding that an empty set accepts nothing. A second vector written to close a count rather than to catch a defect an implementation could plausibly have reports a margin that does not exist, which makes the record worse than leaving the shortfall named.

Checks

966 passed, 1 skipped. Ruff clean across src, tests and examples. The generator reproduces every committed fixture byte for byte, per agentrust-io#171.

Both new vectors were validated against the real verifier before the records were updated: the fifteen other fixture tests passed on the first run, so each produces the failure code it claims. The separation figure and the margin shortfall were each measured and then recorded, not recorded and then assumed.


Generated by Claude Code

…ther two cannot

Continues agentrust-io#116, which asked for obligations 2 and 3 back
"with the separation figure measured and pinned". Three things here, in order of
how much they matter.

## The module's own headline number had drifted

`test_verifier_compatibility_separation.py` exists so the set's separating power
cannot change unnoticed. Its opening paragraph said the set separates "3 of its 8
vectors" while the figure pinned in code twenty lines lower said 4 of 9. A reader
arrives at the docstring first and left with the superseded number.

The figure now lives in one place, next to `SEPARATING`, and the paragraph says
so rather than restating it.

## Two of the four recorded margin shortfalls are closed

`KNOWN_THIN` recorded four refusal rules carried by a single vector each, which
agentrust-io#124 established as insufficient. Two now have a second vector:

`10-superseded-first-in-set-innocent-record-refused` pins
`superseded_profile_in_accepted_set` from the other two directions at once. The
v0.1 identifier is first in the accepted set rather than last, so an
implementation scanning only the tail passes 08 and fails here, and the record
presented is an ordinary v0.2 record with nothing wrong with it. The second half
is what makes it separate where 08 cannot: 08 carries a v0.1 record, which the
schema refuses on its own whether or not a profile rule ran.

`11-empty-profile-string-refused` pins `profile_absent` with a claim that is
present and empty. 07 removes the member outright, so an implementation testing
`"eat_profile" not in record` passes 07 and reads this one as an unrecognised
profile, or as absent and therefore current. It does not separate, and the
reason is structural rather than a flaw: the schema pins `eat_profile` with a
`const`, so any record reaching this rule is already schema-invalid.

Separation measured after: 5 of 11, up from 4 of 9. Vector 10 separates, 11 does
not, both as designed.

## The other two were measured and are not closable

Recorded in `KNOWN_THIN` with the reason, because "measured and shown to have no
second axis" is a different state from "not yet done":

`no_accepted_profiles` fires on the verifier's configuration before any record is
read, and that configuration has one shape. The only other axis, pairing the
empty set with a defect the verifier would catch later, needs `check_freshness`,
which every vector in this set asserts is False for a good reason: a fixed `iat`
would make the set expire.

`superseded_profile_refused` needs a record carrying v0.1 and an accepted set
excluding it, and the set can hold only v0.2, since any other member trips
`unschemaed_profile_in_accepted_set` first.

What is left for both is varying record content, which pins nothing: no
plausible implementation branches on record content when deciding that an empty
set accepts nothing. A second vector written to close a count rather than to
catch a defect reports a margin that does not exist, which makes the record
worse.

966 passed, 1 skipped. Ruff clean. The generator reproduces every committed
fixture byte for byte, per agentrust-io#171.

Signed-off-by: Louielunz <48041247+lywinged@users.noreply.github.com>
@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

Contributor Check: UNKNOWN

Check Result
Profile UNKNOWN
Credential LOW
Overall UNKNOWN

Automated check by AgenTrust Contributor Check.

@github-actions github-actions Bot added the needs-review:UNKNOWN Contributor check flagged UNKNOWN risk label Aug 27, 2026
@lywinged
lywinged merged commit c4af38b into main Aug 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:UNKNOWN Contributor check flagged UNKNOWN risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant