The CONFIRMED loop: LLM self-verification where a text verdict beats a confidence percentage

The CONFIRMED loop is a self-verification protocol: the model generates an answer, then critiques itself, and if the critique finds a mistake, it rewrites the answer. The loop repeats until the critic outputs the signal word CONFIRMED (meaning no errors were found) or a preset attempt limit runs out.

The key finding: if you ask a model to rate its confidence as a percentage, it almost always says something like "80-90% confident" — regardless of whether it's actually right. That's because a confidence number doesn't require real analysis; it's just a habitual phrase. But when you ask the model for a specific text verdict ("confirmed" or "found an error, here it is"), that forces it to actually re-check its logic, and that signal turns out to be reliable 82-88% of the time.

The method runs on a three-step loop: generate an answer → critique it → rewrite if the critique found a problem. And the benefit shows up differently depending on task difficulty: on easy tasks, the loop just stops quickly without any accuracy gain (saving on messages), while on hard tasks it genuinely boosts accuracy, because there the model actually has something worth fixing.

Method Blueprint

STEP 1: Generate an answer to the task → draft answer
STEP 2: Critique this answer line by line → verdict "CONFIRMED" OR a description of the error
STEP 3: If an error was found — rewrite the answer based on the critique → new answer
        Repeat steps 2-3 until CONFIRMED or the attempt limit (D) runs out

All three steps can be run in a single long prompt — the model will play the generator, the critic, and the editor sequentially within one response.

Example Application

Task: A financial analyst is calculating unit economics for a new product — a subscription service with several pricing tiers, discounts, and affiliate commissions. These are calculations with percentages and compound formulas — exactly the kind where it's easy to slip up on one step and not notice.

Prompt:

Task: Calculate LTV and CAC for a subscription service.
"Basic" plan — $9.90/month, 8% monthly churn.
"Pro" plan — $24.90/month, 4% monthly churn.
Marketing spend is $5,000/month, acquiring 300 customers (70% Basic, 30% Pro).
We pay a 15% commission on the first month's payment for affiliate sales.

Step 1 — Answer:
Give a full calculation of LTV, CAC, and the LTV/CAC ratio.

Step 2 — Critique:
Now act as a strict financial auditor. Recheck every formula and every number.
- If the entire calculation is correct — write only the word CONFIRMED.
- If you find an error — point out exactly which step it's in, and give the corrected calculation.

Step 3 — Repeat:
If an error was found, repeat step 2 for the new calculation. Maximum 3 rounds of checking.
Stop as soon as you get CONFIRMED.

Result: The model will produce the first calculation, then explicitly verify each formula like an auditor. If it finds an error (say, in the churn calculation or the plan-weighting), it'll flag it and give a corrected version, then check again. In the end, you get either the word CONFIRMED after a correct calculation, or a final version after a few rounds of fixes.

Why This Works

By default, a model doesn't check its own first answer — it delivers it as final. And if you directly ask "how confident are you, as a percentage?", the model answers with its habitual pattern of polite confidence rather than the result of an actual check — its numeric self-assessment isn't calibrated to correctness.

What the LLM is actually good at is finding specific errors in text when asked not to rate itself but to literally re-check the logic step by step. The role of "critic hunting for a mistake" switches on a level of attentiveness that a confidence-percentage question simply doesn't trigger.

The method works around this weakness by replacing an abstract numeric score with a concrete binary verdict: either an explicit "everything's correct" (CONFIRMED), or a specific pointer to the problem. This works as a reliable stop signal precisely because it demands real analysis rather than guessing a number.

Control levers: - Iteration limit (D) — for routine tasks, set 1-2 rounds; for complex calculations and logic, 3-5 - How you phrase the critique — the more specific your ask ("check every formula," "check every logical step"), the more precisely the model finds errors - Don't use numeric confidence as a stopping criterion — replace it with a text verdict

Prompt Template

Task: {task}

Step 1 — Answer:
Give a detailed solution to the task above.

Step 2 — Critique:
Act as a strict reviewer. Recheck every step of your solution for logic and calculation errors.
- If the solution is completely correct — write only the word CONFIRMED.
- If you find an error — describe it specifically and give the corrected solution.

Step 3 — Repeat:
If an error was found, repeat step 2 for the new solution.
Maximum {iteration_count} verification cycles. Stop as soon as you get CONFIRMED.

Fill in: {task} — what needs to be solved, {iteration_count} — how many rounds of checking are allowed (2-3 for medium complexity, 4-5 for very tangled problems).

🚀 Quick Start — paste this into the chat:

Here's a CONFIRMED-loop template for self-verifying an answer. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[insert template above]

The LLM will ask how many rounds of checking you need and how strict the critique should be — because that determines the balance between the quality of the final answer and the number of messages you'll end up sending.

Limitations

⚠️ Doesn't help on easy tasks: if the task is genuinely simple and the model gets it right on the first try, the loop just burns messages confirming the answer — accuracy doesn't improve, only time is spent.

⚠️ Numeric self-rated confidence is unreliable: if you ask a model to state a confidence percentage, it almost always gives a high number regardless of whether the answer is actually correct. Don't use that number as your stopping criterion.

⚠️ Critique using the same logic can miss a systemic error: if the model made a mistake due to not knowing a fact or having a systemic misconception, a re-check by the same model using the same logic may just repeat the mistake instead of catching it.

How It Was Tested

Researchers took a single "frozen" model (no fine-tuning) and ran the protocol on three task types: logic puzzles (100 questions), grade-school arithmetic (500 problems), and olympiad-level math (500 problems). They compared single-answer accuracy against the self-verification loop, and separately checked whether stopping based on numeric confidence works at all when the model is asked to state a confidence percentage from 0 to 100.

It turned out that at any confidence threshold above 60%, the model stops right after its first answer — it's always "confident enough," even when it's wrong. But the critic's text verdict ("confirmed" or "here's the error") turned out to be a reliable signal — it's what stopped the loop 82-88% of the time without any loss of accuracy.

The most interesting part: on easy logic tasks, the loop added no accuracy at all — it only saved compute by stopping quickly. But on hard olympiad math, where the model's single-shot answer is correct only about a quarter of the time, the loop boosted accuracy by 14 percentage points. That makes sense: self-verification helps exactly where the model is actually capable of spotting and fixing its own mistake — that is, on hard tasks, not trivial ones.

Resources

EvoResearcher — Wei Yu, Suxing Liu, Minjie Yu, Jiahao Wang, Zhijian Zheng, Haocheng Deng, Bing Li (Jiangxi Arts & Ceramics Technology Institute; Universiti Sains Malaysia). Validated on Big-Bench Hard, GSM8K, MATH; backbone models — deepseek-v4-flash and Qwen2.5-72B.