Verify-First: Asking a Model to "Double-Check Its Answer" Often Means Breaking It

Option A — Verify-First (protection against degradation):

STEP 1: Get an answer from the model → regular prompt
STEP 2: Request a check with the verify-first instruction → 
        "re-solve it first, then compare"
        The model changes the answer only if it found a specific error

Both steps happen in one conversation. Repeat Step 2 no more than 1–2 times.

Option B — Self-Consistency (beats iterative edits):

STEP 1: Open 3 independent chats → ask the same question
STEP 2: Compare the 3 answers → pick the majority answer
        (or ask a 4th chat: "here are 3 answers, which one is right?")

These steps need separate chats or requests — the independence is what produces the effect.

Task: A startup founder asked an LLM to calculate unit economics — CAC, LTV, payback period. Got an answer. Wants to be sure it's all correct and asks it to "check the calculations and improve them."

The usual approach (risky):

Check your calculations and fix any errors if there are some.
Improve the answer.

→ There's a >50% chance the model will change correct numbers into wrong ones.

Prompt with Verify-First:

Before changing anything — solve the problem again, independently 
of your previous answer. Don't look at it.

Recalculate the unit economics from scratch: [data]

Then compare the two solutions. Make changes ONLY if 
you find a specific, explicit error in the calculation — and say 
exactly which one. If there are no discrepancies, return the original 
answer unchanged.

Result: The model will show two independent solutions and compare them explicitly. If they match, it returns the original answer with no edits. If it finds a specific discrepancy (say, it mixed up the LTV formula), it points it out and fixes only that part. The odds of randomly breaking correct numbers are close to zero.

The LLM's weakness: A model doesn't store a "correctness flag" for each of its answers. When you ask it to "check," it generates text following the pattern of checking, not the pattern of accuracy. That pattern includes phrases like "Actually..." and "Let me clarify..." — and those trigger changes on their own, whether or not there's an error.

Why verify-first works: The instruction "re-solve it first, independently" breaks that loop. Now the model has two independent solutions — and it compares them instead of just "improving" the first one. If they match, a change has nothing to stand on. Adding "only if you find a specific error" raises the bar for editing.

Control levers: - 🎚 "Only a specific error" — the key phrase. Remove it and the model goes back to changing things out of politeness - 🎚 "Independently of the previous answer" — without this, the model "peeks" at the old answer and drifts toward it instead of honestly recalculating - 🎚 Number of iterations — even with verify-first, don't do more than 2 checks: the gains don't stack - 🎚 Self-Consistency — for important tasks, 3 independent attempts beat 3 iterations of one conversation

Verify-First (built into the check request):

Before making changes — solve the problem again, 
independently of your previous answer. Don't rely 
on it when recalculating.

{task or question}

Once you have a new solution — compare it with the previous one. 
Change the answer ONLY if you found a specific, explicit error, 
with an explanation of exactly what's wrong. If there are no discrepancies, 
return the original answer unchanged.

Self-Consistency (for important tasks):

Solve the problem {task} in three independent ways. 
Give each way its own block of reasoning, 
and don't rely on the previous block for the next one.

After the three solutions — compare the results. 
If all three match, that's the answer. 
If not — find and explain the discrepancy, 
then give the best-justified final result.

Placeholders: - {task} — a specific question: a financial calculation, text analysis, a plan, a problem to solve - {task or question} — the same thing in the verify-first block

🚀 Quick start — paste into the chat:

Here is the Verify-First template for protecting against degradation 
during LLM self-checking. Adapt it to my task: {your task}. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which task you want to check and what answer you already have — because verify-first works as a layer on top of an existing answer and needs context about what exactly to re-check.

⚠️ Tasks without a single right answer: Verify-First is built for tasks with a clear "right/wrong" — calculations, facts, logic. For creative tasks (rewriting text, coming up with a name, generating ideas) the logic doesn't work the same way — there's no "error" to find by recalculating.

⚠️ Doesn't make up for weak models: Verify-First removes degradation but doesn't make the model smarter. GPT-4o-mini with verify-first gives +0.2 pp, while o3-mini without it gives +3.4 pp. Eliminating degradation isn't the same as achieving real improvement.

⚠️ Self-Consistency needs multiple requests: Three independent attempts = three times the tokens. For long tasks, that adds up.

⚠️ Explicitly asking for a confidence self-rating hurts: If you ask the model to rate its confidence (on a 1–10 scale) before solving the problem, accuracy drops by 3.8 pp. The self-assessment distracts from the task itself.

A team from Peking University and the University of Chicago took 7 models — from the fast GPT-4o-mini to the flagship GPT-5 and Claude Opus 4.6 — and ran each through 4 rounds of "self-improvement" on three datasets: 500 grade-school and competition math problems and 200 commonsense questions. The logic was simple: what happens to accuracy if you keep asking a model, over and over, to "check and improve"?

The main instrument is two numbers at each iteration. EIR measures how often the model takes a correct answer and makes it wrong. ECR measures how often it fixes a wrong one into a right one. The researchers were looking for a threshold: at what EIR does self-correction start doing harm?

The most interesting part is the contrast between GPT-5 and Claude Opus 4.6. Similar starting accuracy (96.2% vs. 97.6%), but GPT-5 loses −1.8 pp over 4 iterations, while Opus 4.6 gains +0.6 pp. The reason: GPT-5's EIR is 10 times higher than Opus 4.6's (1.9% vs. 0.2%). That proves it's not about how "smart" the model is, but about a specific mechanism — the ability to leave alone what's already right.

Verify-first was tested as a full experiment: they took GPT-4o-mini, added one instruction, and the −6.2 pp degradation became +0.2 pp. Statistically significant (p < 0.0001). Then they tried the same prompt on models whose EIR was already low — it changed nothing at all. That's exactly what confirms it: verify-first acts surgically, only where there's a problem.

Before making any changes, first verify whether your previous 
answer is correct by re-solving independently. Only change your 
answer if you find a concrete, specific error.

Context: The full wording of the verify-first prompt from the ablation experiment (Section IV-E). This exact text dropped GPT-4o-mini's EIR from 2% to 0% and flipped −6.2 pp of degradation into +0.2 pp. It's short — and that's exactly why it's powerful: there's no loophole for "polite edits."

1. Adaptation: Verify-First for analysis and conclusions (not just calculations)

The principle works anywhere a model can "talk itself out of" a position — investment analysis, legal review, medical interpretation.

💡 Adaptation for data analysis:

Copy

Here is my data: {data or description of the situation}

You already gave a conclusion: {the model's previous conclusion}

Now — forget that conclusion. Analyze the data 
again from scratch. Facts only, no assumptions 
carried over from the previous analysis.

Once you have the new analysis — compare it with the first one. 
Change your position only if the new analysis revealed 
a specific factual discrepancy.

2. Technique: Named agents instead of a faceless "check"

🔧 Technique: two named roles → sharper separation

Instead of "check your answer," give two explicit roles:

Copy

Role 1 — Analyst: solve the problem {task} from scratch, 
without knowledge of the previous answer.

Role 2 — Critic: compare the Analyst's answer with this one: 
{previous answer}. Point out specific discrepancies. 
If there are none, confirm the original answer.

Named roles create a structural barrier against "merging" — the model is less likely to gravitate toward the first answer when recalculating.

3. Extrapolation: Self-Consistency inside a single prompt

For people who don't feel like opening three chats — ask the model to do it itself:

Copy

Solve the problem {task} using three independent methods.

Method 1: [solution]
Method 2: [solution, without peeking at Method 1]  
Method 3: [solution, without peeking at Methods 1–2]

Final: if all three agree, that's the answer. 
If there's a discrepancy, find the error and explain it.

Worse than three truly independent chats (the model still "remembers" the earlier blocks), but better than a single iterative edit.

Paper: When Does LLM Self-Correction Help? A Control-Theoretic Markov Diagnostic and Verify-First Intervention — Aofan Liu (Peking University), Jingxiang Meng (University of Chicago)

Key references: - Self-Refine (Madaan et al., NeurIPS 2023) - Reflexion (Shinn et al., NeurIPS 2023) - Self-Consistency (Wang et al.) - Huang et al. — "LLMs cannot self-correct reasoning without external feedback"