Self-Reflection: Iterative Self-Correction of LLM Answers

STEP 0 (baseline answer):
The LLM generates a first answer to the task → Answer v1

STEP 1 (reflection):
The LLM gets its own Answer v1 + a "review and fix" prompt → Answer v2

STEP 2-N (optional):
Repeat the reflection loop → Answer vN

Every step runs as a separate request to the model. The previous answer is passed explicitly into the next prompt.

Task: Calculate the unit economics of a subscription to an online fitness platform for the US market.

Prompt (round 0):

Calculate the unit economics for a fitness subscription with these parameters:
- Subscription price: $19.99/month
- CAC (customer acquisition cost): $45 via Google Ads
- Churn rate: 8% per month
- Variable costs: $3/user/month (content, support)
- Fixed costs: $50k/month

Find:
1. User LTV (lifetime value)
2. Payback period
3. Break-even point (how many active subscribers you need)
4. ROI over a 12-month horizon

Prompt (round 1 — reflection):

Here's your calculation above. Review it critically:
1. Check the formulas — are there any math errors?
2. Did you account for all costs correctly?
3. Do the final numbers make sense for the US market?

If you find inaccuracies, fix them and recalculate.

Result:

The model will produce two answers:

1. First round: A baseline calculation with possible shortcuts or errors (e.g., the wrong LTV formula when accounting for churn rate)
2. Second round (after reflection): A corrected calculation with more accurate formulas, verified numbers, and detailed explanations for each step

In math and analytical tasks you'll see concrete error fixes: the model finds the wrong formula, recalculates, and shows exactly where it went wrong.

LLM weakness: The model generates text on autopilot following patterns, with no internal "calculator." On the first pass it may:

• Miss details in the problem statement
• Slip up in intermediate steps (especially in long chains of calculations)
• Apply an imprecise formula
• Go with the first solution that comes to mind rather than the best one

LLM strength: The model is great at critiquing someone else's text: it spots logical gaps, checks reasoning steps, and notices missed conditions. When an LLM looks at text from the outside (even its own output), it switches into verification mode instead of generation mode.

How the method leverages that strength: Self-reflection turns a weakness (generating without checking) into an advantage. Round one generates the answer. Round two has the model read that answer as external text, switch on critical thinking, find errors, and rewrite. It uses its verification skills to fix its own hasty generation.

Control knobs:

1. Reflection depth (0, 1, 3+ rounds):
   • 0 rounds → baseline answer, fast and cheap
   • 1 round → 80-90% of all possible benefit
   • 3+ rounds → diminishing returns, overpaying for tokens
   Tune it to the task: Simple questions: 0 rounds. Critical calculations: 1 round. You almost never need more.
2. Feedback mechanism (what to compare against):
   • No feedback: the model reviews by eye
   • LLM-as-a-judge: a second model grades the first
   • External check: run the SQL, execute the code, compare against a reference
   Effect: On SQL tasks, feedback from executing the query improves Claude by 6%, while Amazon Nova does better with no feedback at all or with an LLM judge. There's no universal recipe, so test on your own data.
3. How detailed the reflection instruction is:
   • "Review your answer" → a general check
   • "Check the calculations, formulas, and the logic of each step" → focus on math
   • "Did you account for every condition of the task?" → focus on completeness
   Effect: Specific review criteria produce more targeted corrections.

{task_prompt}

---

[After you get the first answer — second request:]

Here's your answer above. Review it critically:
- {criterion_1}
- {criterion_2}
- {criterion_3}

If you find inaccuracies or omissions, fix them and rewrite the answer.

What to fill in:

• {task_prompt} — your original question/task
• {criterion_1, 2, 3} — specific things to check (e.g., "Check the math," "Did you account for every condition," "Does the conclusion follow")

For complex tasks you can spell out exactly what to check:

Check:
1. Is every input accounted for?
2. Are the formulas and calculations correct?
3. Are there any logical contradictions?
4. Is the result realistic for {context}?

🚀 Quick start — paste into your chat:

Here's a self-reflection template. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which review criteria matter for your specific task, because different domains need a different focus: in math it's formula correctness, in translation it's preserving meaning and tone, in SQL it's syntax and query logic. It'll take the "task → reflection with criteria" pattern and adapt it to your case.

⚠️ Domain-specific effectiveness: In math tasks and sentiment analysis, reflection reliably improves results. In translation and SQL generation, most models get worse, swapping correct answers for wrong ones. Test on your own data before rolling it out.

⚠️ Plateau after 1 round: The first reflection delivers 80-90% of the total benefit. Further rounds barely change accuracy but raise latency and cost 2-3x. For most tasks, the optimum = 1 round.

⚠️ Not equal across models: Amazon Nova Micro, Lite, and Pro get worse at translation with every reflection round. Claude Sonnet 3.5 v2 degrades on SQL tasks after reflection. Mistral Small loses accuracy on sentiment analysis. There's no universal model+method combo, so match it to your use case.

⚠️ Cost/quality trade-off: Every reflection round = an extra API call. For tasks where accuracy is critical (financial calculations, medical analytics), it's worth it. For high-volume processing of simple requests, the extra spend may not pay off even with a 10-15% quality gain.

A team from Amazon and Zalando tested 10 models (Claude Sonnet 3.7/3.5/Haiku 3.5, Amazon Nova Premier/Pro/Lite/Micro, Llama Maverick, Mistral Small/Large) on 4 benchmarks: Math500 (math), Spider (text-to-SQL), IMDB (sentiment analysis), and Flores-200 (translation). Each model ran 0, 1, and 3 rounds of self-reflection, and for Claude 3.7 they also tested built-in reasoning with two thinking-token budgets.

They then built Pareto frontiers of accuracy vs latency: which configuration gives the best result for a given speed or budget. They compared not only final accuracy but also improvement trajectories, tracking how each model flipped correct/incorrect answers from round to round.

Crucially, the team didn't stop at benchmarks. They deployed self-reflection in a production content-localization system at Lounge by Zalando (17 European markets, real copywriters, real marketing campaigns). There they found market-specific effects: in the German market reflection eliminated 100% of errors, in the French market it cut them by 88%, and in the Spanish market by just 39%. That confirmed the main conclusion: there's no universal strategy, and effectiveness depends on the language, the domain, even the specific market within one language.

An interesting design detail: for SQL tasks they added two types of feedback between rounds, the query execution result vs an LLM-as-a-judge assessment. It turned out Claude works better with SQL execution feedback (it sees errors in the data), while Amazon Nova does better with an LLM judge or no feedback at all. Why? An Amazon Nova Pro judge can't give Claude sharper criticism than Claude gives itself, so that outside feedback only confuses it. Concrete SQL results (tables, errors), on the other hand, are objective and help.

The biggest surprise: Claude 3.7's built-in reasoning (marketed as an advanced feature) lost to plain self-reflection in many tests, even though reasoning costs more and doesn't work with prompt caching. That busts the myth that "dedicated reasoning models are always better": sometimes a base model + smart prompt engineering is more effective.

Finding the Sweet Spot: Trading Quality, Cost, and Speed During Inference-Time LLM Reflection

Relevant links from the study:

• Open-source self-reflection implementation for Amazon Bedrock
• Madaan et al. (2023) — Self-Refine: Iterative Refinement with Self-Feedback (the original self-reflection paper)
• Snell et al. (2025) — Scaling LLM Test-Time Compute Optimally

Authors: Jack Butler (Amazon Web Services), Nikita Kozodoi (Amazon Web Services), Zainab Afolabi (Amazon Web Services), Brian Tyacke (Zalando), Gaiar Baimuratov (Zalando)

Presented at: NeurIPS 2025 Workshop: Efficient Reasoning