Self-critique and self-refine: what the research shows

What self-refine is

The basic loop has three steps: generate a draft, critique it, revise it using the critique. The variants differ in where the critique comes from and how tightly it is scoped:

• Staged in one prompt: draft, then a checklist audit, then a fix, as in Re5 and GEIS.
• Separate requests: generate, score against each principle, revise only what failed, as in Reflect.
• Role-based critics: the same model critiques as a CFO, a lawyer or a skeptical customer. In the generate-critique-revise study, diversity of critic roles drove the feasibility of ideas, while diversity of generator roles drove novelty.
• External feedback: a compiler, a test, a validator or a human list of issues replaces the model's own judgment.

Why critique after writing can work

Writing and checking load the model differently. While generating, it has to hold every requirement in mind at once; while checking, the finished text and the rule sit side by side. The Reflect authors found models were better at spotting principle violations in finished text than at avoiding them while writing, and the post-hoc check cut the share of serious violations by 2-3x on GPT-4-Mini, Claude-Haiku-3.5 and Mistral-7B.

The gain is largest for weaker models on tasks with checkable answers. In a study of 10 models, one reflection round took Nova Micro from 22% to 71% on math, and the first round delivered 80-90% of the total benefit. The same study found most models got worse at translation and SQL after reflection.

The critique has to be specific

The most consistent finding across these papers: models can fix errors much better than they can find them.

• RefineBench (1,000 tasks, 34 models) found "improve it" stalled near 30% quality, while a list of specific issues reached 90-98% in five rounds. Claude-Opus-4.1 reached 98.4% with guidance and 19.5% without; giving only a checklist of what a good answer contains lifted Gemini 2.5 Pro from 31% to 76%.
• Think² measured the gap directly: models noticed an error 51% of the time, diagnosed it 28% and corrected it 16%. Planning, monitoring and evaluation phases raised diagnosis to 58% and correction to 50%, but on Llama-3-8B, which lacks a reasoning mode, GSM8K accuracy fell from 80% to 69%.
• Precise external feedback beats general guidance. In compiler-guided refinement, GPT-5 solved 30 of 56 Idris problems with documentation, 34 with a reference guide and 54 with actual compiler errors pasted back. Iterative self-repair reports 45% versus 77% success depending only on how precisely the error was described.
• How you show the error matters. Feedback inversion found that quoting the failed attempt verbatim raised the chance of repeating it from 6% to 54%; describing the problem in your own words avoided the copying. Conversely, re-submitting one dubious claim as a new message and asking whether it is correct gave the model a concrete target and corrected 60-90% of errors it otherwise missed.

Where self-refine plateaus or breaks

• Diminishing returns. Self-correction dynamics found accuracy plateaus or drops after 1-2 iterations: open answers drift, multiple-choice answers barely move. In Agentic XAI, explanation quality rose 30-33% over 3-4 rounds, then fell by 60-80% as answers filled with generic text.
• Breaking right answers. In the Verify-First study, GPT-4o-mini lost 6.2 points over four rounds of "check and improve" because it changed correct answers more often than it fixed wrong ones. The gate described in DISC addresses the same failure: a judge must find concrete evidence before anything is edited; switching the gate off raised degradations 2-3x.
• Ownership bias. Models rate their own answers with 16-27% higher confidence than the same text attributed to a user, and an LLM judge approved bugs in code that looked right. Presenting the draft as someone else's work, or reviewing it in a fresh session, removes the authorship cue.
• Contextual drag. A wrong draft left in context pulls the next attempt toward the same structure: accuracy dropped 10-20% for strong models and up to 50% for weak ones, and saying the draft was wrong barely helped.
• Missing knowledge. Review cannot add facts the model lacks. In a code-generation study, self-critique gave 0% improvement on projects that needed external API knowledge, while adding documentation reached 100%.
• Independent attempts can win. Mixture-of-Agents found iterative self-refinement worse than a plain chain-of-thought answer at every budget, while independent answers plus one synthesis gained 7.1 points.

How to apply it

1. Write the criteria before you critique: a checklist of what a good answer contains, or the principles it must follow.
2. Ask the model to score each criterion separately and revise only what failed; tell it to leave the rest unchanged.
3. Feed back precise errors (a failing test, a compiler message, a named problem) rather than "make it better", and describe the error instead of pasting the broken attempt.
4. Stop after one or two rounds unless an external check shows progress.
5. For checkable answers, re-solve independently before changing anything, or start a fresh chat; for drafts that need new facts, supply sources instead of another review.

For checking a single answer's claims see verification prompts; for independent answers instead of revisions see self-consistency.

Step 1:
Here are the principles you must follow:
{list_of_principles}

{your_task}

Step 2:
Score your answer on each principle from 1 (fully violates) to 5 (fully complies).

Format:
Principle 1: [score]/5 - [short justification]
Principle 2: [score]/5 - [short justification]
...

Step 3 (if any scores are below 3):
Low-scoring principles: {list_of_violated_principles}

Critique your answer against these principles.
Then rewrite the answer based on the critique.

Format:
CRITIQUE: [what exactly was violated and why it's a problem]
REVISION: [improved answer]

Here is the {content_type}:

{text}

Evaluate it against this checklist (answer Yes/No for each item):

1. {criterion 1}?
2. {criterion 2}?
3. {criterion 3}?
4. {criterion 4}?
5. {criterion 5}?

For every "No", write what specifically needs fixing.

---

Revise the {content_type} on these points:

1. {specific problem 1} - {what to do}
2. {specific problem 2} - {what to do}
3. {specific problem 3} - {what to do}

Don't change anything else.

Check the following {text_type}. Follow the roles strictly - don't skip any.

ORIGINAL TASK: {task}

TEXT TO CHECK:
"""
{text}
"""

ROLE 1 - VERIFIER:
Write {number_of_questions} targeted questions about specific claims above.
Answer each question on your own, as if you had never seen the text above.

ROLE 2 - JUDGE:
Compare each answer with the corresponding claim in the text.
A direct contradiction is required, not a subjective quality judgment.

Issue one of two verdicts:
NO_ERROR - return the text unchanged and explain the conclusion in one sentence
ERROR - quote the exact spot of the error + state which evidence confirms it

ROLE 3 - CORRECTOR (only if the verdict is ERROR):
Fix ONLY the quoted spot.
Copy the rest of the text word for word, unchanged.
Output the final text.