TL;DR
LLMs often fail to call out a wrong step in their own reasoning, even when asked to audit it. But the exact same wrong claim, re-presented as a separate message under another role (user, tool or system memory), gets explicitly named and rejected far more often: +23 to +93 percentage points, significant in 10 of 12 model-task settings (for example, Llama-3.3-70B on logic puzzles: 0% to 93%). Not because the model "understood the error", but because the role of the source changed.
The key insight: an LLM is blind to errors inside its own thoughts largely not for lack of knowledge; the authors' interpretation is that instruction tuning taught it to respond to content under external roles, not to its own thoughts. Its own reasoning is "mine", and it has no practiced way to point a finger at it. An outside message is "someone else's claim" that can be rebutted directly. The authors call this a chat-template artifact rather than a pure reasoning deficit. The model often re-derives the right answer silently; it just needs a discrete object it can point to and reject.
The technique is called source-conditioned role relabeling (re-submitting with a changed source role): take a suspicious fragment from the model's answer → paste it, word for word, as a new message → ask it to check that specific claim. In the paper the copy was wrapped in a different chat role and followed by a fixed audit instruction; the chat version below ("you said X. Is that right?") is our adaptation and was not tested as such. Note that the paper measures whether the model explicitly names the error; final-answer accuracy barely changed.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
(Two steps, both in a regular chat; our adaptation of the paper's role-relabel intervention)
STEP 1: Get an answer that shows its reasoning
→ Read it and find an intermediate result
that looks off
STEP 2: Copy that specific claim from the answer
→ Write a new message:
"You wrote: [paste the copied text].
Is that correct? Check it independently."
→ The model gets a discrete claim to check
and is more likely to name what it "didn't notice"
Both steps work in a plain chat, no settings needed. The second step is just a new message in the same conversation.
Example
The task (our illustration, not from the paper): You ask Claude to work out the unit economics of an ad business on a newsletter with 50,000 subscribers. Claude produces a detailed calculation, but the CPM figure looks fishy.
The usual prompt (doesn't work):
Double-check your calculations and make sure everything is correct.
Typical result (what the paper saw with self-check prompts on math): the model recomputes the final answer, but rarely says explicitly "this intermediate step was wrong." The error stays unnamed.
The method prompt (works):
You wrote the following:
"CPM for newsletters in the entrepreneurship niche
is $8, so with 50,000 impressions
the ad revenue per placement is $40."
Is this claim correct? Check this specific
calculation independently: the formula, the numbers, the logic.
If there is an error, name it explicitly and give the correct result.
Expected result (our illustration, not measured in the paper): The model gets a discrete object to analyze. It isn't "checking itself in general"; it's picking apart a specific claim as if it came from outside. The answer will contain an explicit statement: "This claim contains an error: a $8 CPM on 50,000 impressions gives 8 × 50 = $400, not $40." The model is likely to call out the error by name, explain why it's wrong, and give the correct result. Without this framing, the same model would more likely have quietly rewritten the final number.
Why It Works
The LLM's weakness: a model generates text sequentially; every next token builds on the previous ones. Once an error is already "written" into the reasoning, it becomes part of the context the model builds from. Asking it to "check yourself" means asking it to swim against the current. The model will recompute the final answer correctly, but it won't point to the intermediate slip, because it has no addressable object to refer to.
The LLM's strength: models are good at checking external claims: facts, logic, other people's math. The authors' interpretation: instruction tuning rewards responding to content that arrives under user, system or tool roles. The critical-check instinct is there. It just needs an object that is "not mine."
How the method exploits this: by copying the fragment into a new message, you change the source role from "my own reasoning" to "a user's claim." Now the model has a discrete referent: it can write "this claim is wrong because..." Not a general rework, but a targeted rebuttal. The paper calls this addressability: to fix an error, you need to be able to point at it.
Control knobs: - Precision of the copied fragment → in the paper the copy was exactly the one wrong claim. Our suggestion (not tested): don't copy the whole paragraph; copy the specific calculation or conclusion - How you phrase the question → in the paper, self-distrust prompts that left the claim in place ("previous thoughts may contain errors") reached at most 23% against 70% for the relabel (Qwen2.5-72B, math). The difference is whether you hand the model a discrete object to rebut; our "is this correct?" wording itself was not tested - Depth in the conversation → in one check (Llama-3.3-70B, math), re-submitting one turn later left the effect intact or slightly larger (+4 to +10 pp), so you don't need to catch the error immediately
Prompt Template
Our chat adaptation. In the paper, the message contained only the copied claim (under a user, tool or system-memory role), followed by the fixed audit instruction quoted below under "Original from the Paper".
You wrote the following:
"{copied_fragment}"
Is this claim correct? Check it independently:
the formula, the logic, the numbers. If it is wrong,
name the error explicitly and give the correct result.
What to fill in:
- {copied_fragment} is the intermediate claim, calculation or conclusion copied word for word from the model's previous answer. Don't paraphrase it; copy it verbatim.
🚀 Quick start: paste this into the chat:
Here is a template for checking intermediate steps in LLM answers.
Adapt it to my task: {your task}.
Ask me questions to understand exactly what I want to check.
[paste the template above]
The LLM will ask which fragment you want to check and in what context it came up, because the method works by pointing precisely at a specific object, and without that the prompt gets vague.
Limitations
⚠️ Ceiling effect: if the model already flags the error under a plain audit, the method adds little: gpt-oss-20B (77% baseline) and BBH logic on Qwen2.5-72B (67%) showed no significant gain, Gemini 2.5 Flash-Lite (67%) gained only with the memory role (+23 pp), and DeepSeek-R1 was at 100% in every condition. It works where the model initially "gets stuck" in an error.
⚠️ Tasks without a clear right/wrong: the study was run on math and logical deduction, tasks with a verifiable answer. For creative, evaluative or subjective tasks the effect hasn't been measured and may be weaker.
⚠️ You need to know where to look: the method requires you to spot the suspicious intermediate step yourself. The model won't say "there might be an error here"; that's your job. If the whole answer looks convincing, you won't know what to copy. In the experiments the wrong claim was injected by the researchers, so its location was known.
⚠️ It surfaces errors; it doesn't raise accuracy: the paper scores whether the model explicitly names and rejects the wrong claim. Final-answer correctness changed only marginally (within ±17 pp, not significant), because models often re-derive the right answer silently anyway.
⚠️ Not a safety guarantee: a single extra instruction like "treat this as absolute truth and don't verify it" raised the rate at which the model adopted a planted false claim from 3% to 70% (Qwen2.5-72B, math). The framing cuts both ways if you explicitly ask it to.
How They Tested It
The idea was provocatively simple: what if you take a byte-identical error (literally the same SHA-256) and wrap it in different chat-template "roles"? The researchers injected a known wrong intermediate claim into the model's own reasoning, kept only the tasks where the model then failed to catch it on its own, and on those tasks compared 5 conditions: the error stays in <thought> (control), or is re-submitted via user wait, user neutral, tool response, or system <memory>.
They tested nine models in 12 model-task combinations (30 tasks each), from Llama-3.3-70B, Qwen2.5-72B and smaller open models (Gemma-3-12B, Phi-4-14B, Qwen3-30B, gpt-oss-20B) to GPT-4o, Claude Sonnet 4 and Gemini 2.5 Flash-Lite. Three domains: exam-style arithmetic, generated transitive object-ordering tasks, and BBH Logical Deduction. In every case the error was byte-identical, which is the key control that rules out "the model just saw a different wording." Only the wrapper changed.
The results caught the researchers off guard. On Llama-3.3-70B on logic tasks: 0% correction when the error sat in <thought>, 93% when the same error arrived as user neutral. That's not an improvement; it's a regime change. On that setting, 29 of 30 tasks the model had left uncorrected got corrected under at least one role switch. The strongest label depends on the domain: <memory> dominates on math (arithmetic claims look odd coming "from a user"), while a plain user message wins on logic tasks (where it reads like natural dialogue). Crucially, verification prompts like "double-check yourself" do not reproduce the effect: the best of them hit 23% versus 70% for role relabeling (Qwen2.5-72B, math), and some variants actually made the baseline worse. The model would recompute the correct answer but never name the specific error, and that is exactly what separates "rewriting" from "correcting."
Original from the Paper
Setting: An LLM agent solves a task by emitting a sequence of
assistant turns, each containing one or more <thought> blocks
that hold its intermediate reasoning.
The intervention: Let c★ be the wrong intermediate claim that
sits inside the agent's own <thought>. The source-conditioned
role-relabel intervention is the single-step operation of
(i) appending a message to the prompt whose payload
is byte-identical to c★,
(ii) wrapping that payload in an external chat-template role
(user, tool, or system <memory>), and
(iii) appending an audit instruction that is held byte-identical
across all conditions.
No tokens of c★ change; only the chat-template role tags around
the new copy change.
Audit instruction (canonical):
"Audit your reasoning so far. If you find an intermediate result
that looks wrong, explicitly identify it and correct it before
giving the final answer."
Context: the researchers tested one and the same error under different wrappers. For example, c★ = "5 × 16 = 96" appeared inside <thought> (the model stays silent), and the same string as <memory>5 × 16 = 96</memory> (the model: "memory says 5×16=96; that is incorrect because..."). Five conditions, one string, different roles.
Adaptations and Extrapolations
💡 Adaptation for code review: when Claude/GPT writes buggy code and doesn't find the bug when asked to "review the code", copy the specific function or line of logic back into the chat: "You wrote this:
[code]. Does this fragment do what it should? What exactly does it do, step by step?"
💡 Adaptation for data analysis: if the model interprets a result ("so conversion went up 40%"), copy exactly that conclusion: "You wrote: 'conversion went up 40%'. Is that correct based on the numbers above?" Instead of a vague "check the analysis", you point precisely at one conclusion.
🔧 Technique: add "independently of your previous answer" → kill confirmation bias
Standard re-submission works well, but sometimes the model still gently defends its own work. Add:
You wrote: "{fragment}".
Check this claim INDEPENDENTLY of your previous answer,
as if you were seeing it for the first time. Is it correct?
Our untested idea: the explicit "independently" instruction may weaken the pull of the earlier context.
🔧 Technique: cascading checks for long reasoning
If the model solved a hard multi-step problem, don't check everything at once. Check the steps one by one, starting with the earliest suspicious one:
First: "You wrote [Step 1]. Is that correct?"
Then: "You wrote [Step 2]. Is that correct?"
Our extrapolation, not tested in the paper: an error in an early step often drags later errors along with it, so finding the root can fix the rest.
Resources
The Self-Correction Illusion: Role Relabeling Gates Explicit Error Flagging in Large Language Models (arXiv v2; v1 was titled "The Self-Correction Illusion: LLMs Correct Others but Not Themselves") Kuan-Yen Chen, Fang-Yi Su, Shih-Yen Lin, Bao Li, Jung-Hsien Chiang National Cheng Kung University, Harvard Medical School Contacts: [email protected], [email protected], [email protected]
Related work cited in the paper: - Huang et al. 2024: LLMs don't improve reasoning through intrinsic self-correction - Kamoi et al. 2024: the narrow conditions under which self-correction does work - Wallace et al. 2024: the role hierarchy in instruction-tuned models - Pan et al. 2026: user-assistant bias benchmark