Record what a run observed rather than what it predicted - #556
Merged
Conversation
Two findings from the review on the promotion, and the second is the one that matters. The Blog audit report closed a paragraph with a prediction, that the same four findings "will appear against every fleet repo the moment that change promotes". A report records what a run observed. It also turned out to be wrong: the change promoted, the findings were traced to spec/audit.py asserting a bypass list the config had deliberately stopped managing, and #551 removed that assertion, so the four do not appear. A prediction in a report is stale the moment reality diverges and nothing re-checks it, which is the same failure the driftNote rules already name in the registry. The sentence now ends at the reason the findings were not counted and hands off to the enumeration that follows. The PENDING_MARKERS comment put its relative clause next to the wrong noun, so "a registry driftNote that assert" read as a disagreement even though the subject is the plural phrases. Reordered so the clause sits with what it describes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates audit/report wording to record what an audit run observed rather than making forward-looking predictions, keeping reports verifiable by re-running the same audit.
Changes:
- Rewords
reports/blog/audit.mdto remove a future-tense prediction and instead summarize the re-run’s observed results. - Rephrases a comment in
spec/audit.pyso the relative clause clearly modifies “phrases” (not “driftNote”).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| spec/audit.py | Clarifies wording in the PENDING_MARKERS comment without changing logic. |
| reports/blog/audit.md | Removes an unverifiable prediction and frames the paragraph as an observation from the re-run. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both findings from the review on promotion #555. A promotion's head is
develop, so the fix lands here and #555 picks it up.The report predicted, and the prediction was wrong
reports/blog/audit.mdclosed a paragraph with:Two things wrong with that, and the second is why it matters.
A report records what a run observed. A prediction cannot be verified by re-running the audit, which is the one check the run-stamp discipline offers a reader.
And it turned out to be false. The change promoted. The four findings were then traced to
spec/audit.pyasserting abypass_actorslist thatconfigure.shhad deliberately stopped managing, and #551 removed that assertion. So they do not appear against every fleet repo. Verified:spec/audit.py Blogreports 0 defect/letter/error on this branch.A prediction goes stale the moment reality diverges and nothing re-checks it. That is the same failure the registry's
driftNoterules already name, arriving in a report instead — and it is the second instance today of a document asserting a future that stopped being true.The sentence now ends at the reason those findings were not counted, and hands off to the enumeration that follows it.
The comment attached its clause to the wrong noun
The subject is the plural phrases, so
assertwas correct, butdriftNote that assertsits adjacent and reads as a disagreement. Reordered so the clause sits with what it describes. This is the second pass over these two lines today, the first having added a missing verb.Verification
Diff-scoped
prose_lintclean, editorconfig clean, CRLF preserved on the report,spec/audit.pyparses, and the audit still reports Blog at zero findings.