Skip to content

test(claude-review): enforce first-attempt/retry prompt-block parity #341

Description

@kyle-sexton

claude-review.yml requires the first-attempt and retry steps to carry byte-identical prompt: blocks, and says so in a comment ("the two must not diverge"), but nothing enforces it. The sibling lane already has exactly this test.

Current state

  • claude-review.yml — invariant stated in a comment only. .github/scripts/claude-review-superseded-guard.test.cjs has four tests, none covering parity.
  • claude-security-review.yml — enforced. .github/scripts/claude-security-review-fail-closed.test.cjs:417 asserts with-block equality across the two attempt steps, plus action-pin parity, continue-on-error, and the retry if:.

Why it matters

The two prompt blocks are maintained by hand and are long. A change to the first attempt that misses the retry produces a lane whose retry reviews under different instructions than the first attempt — and the divergence is invisible until someone diffs the blocks manually. Every PR touching this lane's prompt currently relies on the author remembering, and on a reviewer checking by eye.

This was checked by hand during #338 (both blocks identical, 810 bytes) precisely because no test does it.

Suggested fix

Mirror the security lane's assertion into claude-review's test file. The security lane's version compares the whole with: map rather than just prompt, which also catches pin drift and a diverging timeout-minutes — worth keeping that shape rather than narrowing it to the prompt string.

Not folded into #338: that PR is a behavior fix to the lane's arg composition, and this is test coverage for a pre-existing invariant it happened to surface.

Context: #338.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageNot yet classified. Floor until a type and one priority tier are set.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions