On google/osv-scanner #3107 the maintainer asked: "Do you have a reproduction with real world advisories using the CLI or library?" The reply said "Yes" and pasted four real advisories fed to an internal package function. The maintainer's answer: "That's not a reproduction though that uses the public API". The reply passed claim-check and the adversarial review because every fact in it was true; nothing checked whether the facts answered the question.
The gate that was missing: a reply to a maintainer question is checked against the question, condition by condition, before it is posted.
Proposed:
review.sh brief --question FILE takes the maintainer's comment verbatim. The brief tells the reviewer to split the question into every condition it names (each noun such as "CLI", "library", "real world", "public API" is one) and to report, per condition, the exact evidence the reply carries. A condition with no evidence is a finding that blocks POST AS IS.claim-check.sh gains the same decomposition when given --question FILE: it prints the conditions it finds and refuses a reply that opens with "Yes" or "No" when any condition has no matching text.- The skill's step 10 says what a reproduction is: the shipped command line or an exported function of a public package, run on a fresh clone, with its real output pasted. A call into an internal package is not one.
Ledger row: docs/lessons.md 2026-09-27.
On google/osv-scanner #3107 the maintainer asked: "Do you have a reproduction with real world advisories using the CLI or library?" The reply said "Yes" and pasted four real advisories fed to an internal package function. The maintainer's answer: "That's not a reproduction though that uses the public API". The reply passed claim-check and the adversarial review because every fact in it was true; nothing checked whether the facts answered the question.
The gate that was missing: a reply to a maintainer question is checked against the question, condition by condition, before it is posted.
Proposed:
review.sh brief --question FILEtakes the maintainer's comment verbatim. The brief tells the reviewer to split the question into every condition it names (each noun such as "CLI", "library", "real world", "public API" is one) and to report, per condition, the exact evidence the reply carries. A condition with no evidence is a finding that blocks POST AS IS.claim-check.shgains the same decomposition when given--question FILE: it prints the conditions it finds and refuses a reply that opens with "Yes" or "No" when any condition has no matching text.Ledger row: docs/lessons.md 2026-09-27.