Skip to content

The whole-unit review loop has no committed stop condition, so it does not converge #1267

Description

@ptr727

The whole-unit canonical pass has no committed stop condition, so the loop runs until an agent decides on its own to stop. Measured repeatedly, it does not converge, and the cost is large and invisible until it has been paid.

Evidence

  • PR Settle Seven Section 4 Contract Claims the Workflows Falsify #1198 ran eight fix-and-re-read rounds over sixteen units. Nearly every round's findings were defects the previous round's fixes introduced, not defects in the original text.
  • The Promote Develop to Main: Settle Seven Section 4 Contract Claims the Workflows Falsify #1203 chain ran eleven whole-unit passes and five diff passes across two fix branches. One sentence in a condensed D4.5 item took six revisions.
  • In a single session driving several branches in parallel, three separate agents each ran past 150 tool calls and 200,000 tokens, most of it whole-unit passes returning ten to thirteen findings that were overwhelmingly pre-existing rather than introduced by the branch. One spent five review rounds on a single sentence while holding the only lock on the skills tree, blocking four queued issues behind it.

Why the obvious stop rule does not work

"Ship when the round finds only pre-existing defects" never terminates, because every pass finds something and the classification is the agent's own. A finding count is not a stop condition.

Two framings do terminate, and neither is written down:

  1. A ship or no-ship question rather than a finding count. "Is any false or misleading claim remaining?" closes; "did you find anything?" does not. Style preferences get listed separately, under their own label, and do not block.
  2. A category rule fixing scope rather than volume. A finding on text the diff wrote is fixed; a finding on pre-existing text is filed; a style preference on text already rewritten twice takes the shortest defensible wording and stops.

The second needs two named carve-outs or it swallows real work: a finding load-bearing for a maintainer decision, and a finding showing the branch's own change introduced a false precondition, are both in the first category even when they look pre-existing.

What the rule already says, and what it does not

local-strict-review states that a recorded pass records that a review ran, never that the content is clean, and that findings are advisory. That is the premise a stop rule would rest on, and it is already committed. What is missing is the consequence: a clean verdict plus a list of style preferences is a ship, not another round.

What a fix would have to address

  • Where the stop condition belongs, given GOVERNANCE.md "Verification Discipline" keeps the rule and local-strict-review keeps the procedure.
  • A round budget stated as a number rather than left to judgment, since every agent that has run this loop has exceeded it while believing it was being thorough.
  • Costing, so the budget can be reasoned about: roughly four minutes per whole-unit pass, so sixteen passes is an hour of wall clock on one small prose change.
  • Whether the budget differs for a change to a contract document versus to code, since the evidence above is almost entirely prose.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    proseA defect in rule or procedure text

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions