Verification prompts: how to make an LLM check its answer without breaking it

What verification prompts are

A verification prompt is a separate, structured check of claims or reasoning, run as its own step instead of a trailing "double-check your answer". It names what is being checked (atomic claims, conflicts in the input data, each reasoning step, the question the model actually answered) and what the check must return (supported or refuted, a quote, an error type, a verdict).

It overlaps with self-refine, but the goal differs: self-refine improves a draft, verification decides whether to trust it. Most of the studies here measure the second, and several show that a vague request to re-check does more harm than good.

Why a separate, structured check works

Solving and checking are different tasks, and models do the second far better when it is asked for separately. In the Contradiction Gate study on 91 tax cases, models given contradictory facts produced an answer as if the data were clean in 63-76% of cases, yet the same models caught 75-83% of those contradictions when a separate request asked them to compare the facts instead of calculating. Reasoning modes did not close the gap; the framing of the request did.

Structure is what makes the check real. Asked point-blank whether an answer contained a hallucination, GPT-3.5 reached an F1 of about 33% on question answering in the HalluClean study; a plan, execute, verdict, correct sequence with a one-line task router raised it to 68%, and the router alone moved summarization from 25% to 60%. The specification-grounding study found the same for reviews: an open "find problems" invented issues about a third of the time, while checking a numbered list of rules one by one brought false alarms to zero.

Order matters too. A model that reads a convincing answer first anchors on it. In the verification-asymmetry study, writing "My answer: ..." before comparing cut false positives from 72% to 1.2%, where stricter judges and ensembles of three model families had failed.

Variants that tested well

1. Atomic fact-checking. Break the answer into one-sentence claims, each with a basis, a way to check it and a status. In a randomized trial with 356 physicians, AFC raised trust in GPT-4.5 oncology recommendations from 26.9% to 66.5%: matching one claim to one source is easier than judging a paragraph.
2. Verify a proposed answer before solving. Verification-First adds a line like "Proposed answer: 1. First verify this answer, then give the correct one." Checking an outside answer avoids defending one's own; it added 5-7% on GSM8K math and only 2-3% on the knowledge-heavy GPQA.
3. Two different modes must agree. CoT-PoT solves once in words and once in executed code; when both agreed, the answer was correct with probability of at least 0.991 for every model tested, and 78.6% of problems were settled with two requests.
4. Reconstruct the question. Trace Inversion asks the model which question its reasoning actually answered and compares that with the one you asked, catching silent rewrites of the task.
5. Retell the formal result. The backward-translation check has the model retell a contract, spec or formula in plain words, where gaps between intent and output are easy to spot; direct translation into formal text hit an 89% accuracy ceiling.

When it helps and when it hurts

The clearest failure is the generic re-check. In the self-correction diagnostic study, repeated "check and improve" rounds broke correct answers more often than they fixed wrong ones: GPT-4o-mini lost 6.2 percentage points over four rounds. One instruction, re-solve independently and change the answer only for a concrete, specific error, turned that into +0.2 pp; three independent chats with a majority vote gave +2.2 pp where three in-chat rounds gave -4.6 pp. Asking for a confidence rating before solving cost 3.8 pp.

Agreement is not verification either. In the Refute-or-Promote vulnerability hunt, more than 80 agents unanimously confirmed an OpenSSL vulnerability that one short test disproved; reviewers with a mandate to "kill" the finding in fresh chats, a critic from another model family and an empirical test removed about 79% of candidates before disclosure. A check also inherits what it is shown: in the relay-not-check study, auditors given an earlier agent's conclusion repeated it 96% of the time, and removing that conclusion raised accuracy on the cases where the earlier agent was wrong from 4% to 45%, at a cost on cases where it was right.

Limits the studies agree on: a verifier cannot check facts it does not know, so pair it with search or a source document; it adds at least one request per answer; and it can raise false alarms, as when the Contradiction Gate cost up to 5 percentage points of accuracy on clean data. Use it where an error is expensive and the claim is checkable.

You are checking {content_type} for factual errors and hallucinations.

{CONTENT_TYPE} TO CHECK:
{text_to_check}

STEP 1: Draft a verification plan. Which facts need to be verified? What could the sources of error be?

STEP 2: Carry out the plan step by step. Check each fact and show your reasoning.

STEP 3: Draw a conclusion: does the text contain hallucinations? Yes or No.

STEP 4: If yes, rewrite the text with the errors fixed.

REQUEST 1 (solving):
You are a {expert role}. Here's the input data:

"{data/brief/task description}"

{specific instruction — calculate/solve/draw a conclusion}.
If there isn't enough data to answer — write "cannot answer."

---

REQUEST 2 (verification, as a separate message):
Check this same input data for internal consistency:

"{same data/brief/task description}"

Are there any facts in it that contradict each other?
Answer with one of three options:
1) "everything is consistent"
2) "missing data" — specify exactly what's missing
3) "contradiction found" — name specifically which two facts conflict

---
If the answer to REQUEST 2 is "contradiction found," do not use the answer
from REQUEST 1. Instead, state that the data is contradictory
and specify exactly what needs to be clarified.

Before making changes — solve the problem again, 
independently of your previous answer. Don't rely 
on it when recalculating.

{task or question}

Once you have a new solution — compare it with the previous one. 
Change the answer ONLY if you found a specific, explicit error, 
with an explanation of exactly what's wrong. If there are no discrepancies, 
return the original answer unchanged.