Convergence is not correctness

Bet 10 / Loop engineering

A loop always converges. That tells you nothing about correctness.

An agentic loop retries until a verifier says "done", so it converges on whatever passes the check, not on what you meant. Whether "done" is actually correct is set by the verifier, not by convergence. Two forces pull against each other: refinement makes the loop genuinely better, while optimizing against a weak verifier games it. Watch which one wins.

Base correctnessiHow often a fresh candidate is actually correct, before the verifier looks. Task difficulty against model capability.

Refinement / iteriHow much the loop genuinely improves each iteration: candidate correctness rises by this much per pass.

Verifier false-acceptiHow often the verifier passes a wrong answer. A false-accept ends the loop on an incorrect output, and it is what the loop selects for.

Verifier false-rejectiHow often the verifier rejects a correct answer. Higher means the loop runs longer, but does not make done more trustworthy.

Gaming / iteriReward hacking: how much the loop learns to exploit the verifier each iteration, raising its false-accept rate. The RLVR failure mode.

Iteration budgetiHow many times the loop may retry before giving up. More iterations raise the chance of reaching done, not that done is correct.

Loops that converged

—%

Of those, correct

—%

"Done" but wrong

—%

Verdict

—

converged (looks done) correct given done (the truth) over iterations

Outcome of every loop at the budget:

How this is computed

Exact, via the loop's stopping-time distribution. Each iteration t, a candidate is correct with probability pₜ = base + refinement·(t-1); the verifier's false-accept rate is faₜ = fa₀ + gaming·(t-1). A correct candidate passes with probability 1 − false-reject; a wrong one passes with faₜ. The loop stops on the first pass.

P(correct | done) is the verifier's precision at the base rate: p(1−fr) / (p(1−fr) + (1−p)·fa). It does not depend on how many times you loop.
More iterations raise convergence toward 1, and nothing else. On a weak verifier, they raise the chance "done" is a false-accept.
Refinement lifts correctness over iterations; gaming collapses it. The verifier, not the loop, decides which wins.

The argument of Loop Engineering and Bet #10: the verifier is the part that actually decides.

Interactive tool · built by subhadipmitra.com ↗