TL;DR
Thought-ICS is a self-correction method for LLMs built on step-by-step generation of reasoning. Instead of generating text in one continuous stream (like regular Chain-of-Thought), the model produces its reasoning one thought at a time: at each step it deliberately states the next complete thought, pauses, then moves on. This creates a discrete reasoning chain with clear boundaries between steps. After generation the system checks the answer, and if it's wrong, it starts a correction loop: the model analyzes its steps, finds the first faulty thought, rolls back to the last correct step and generates an alternative continuation.
LLMs are bad at finding errors in a continuous block of their own reasoning. When a model generates CoT the usual way (token by token), you get an unbroken stream of text with no explicit boundaries between logical steps. Trying to find the error becomes "point to exactly where in these three paragraphs you went wrong," and the model can't pinpoint the problem: it often points past the first error or misses entirely. The researchers showed that on unstructured CoT, models land on an erroneous prefix (a prefix that still contains the error) in a large share of cases, which breaks the whole idea of rolling back and resampling.
A discrete structure solves the problem. When the model generates one thought at a time, each step is a deliberate decision about "what the next complete thought should be." That creates natural decision boundaries in the reasoning. During analysis, the model can walk through the numbered steps and say exactly "step 6, here's the error." Rolling back to step 5 gives a clean prefix (one with no errors). Large models land on a clean prefix 97% of the time. From a clean prefix, resampling works several times better: the model builds alternative reasoning on a correct foundation. Thought-ICS shows a 20-40% lift over baseline Chain-of-Thought when a verifier is available.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
GENERATION (Thought MDP): One thought at a time until the final answer
Prompt: "State the next reasoning step (one observation, calculation or deduction)"
→ Thought 1 [stop]
→ Thought 2 [stop]
→ ...
→ Thought N [final answer]
VERIFICATION: Is the answer correct?
NO → continue the correction loop
YES → exit
LOCALIZATION: Find the first faulty thought
Prompt: "Analyze each step. Find logical flaws, incorrect calculations,
false assumptions. Return the number of the first faulty step"
→ "Error in step 6: [explanation]"
RESAMPLING: Roll back and branch
Roll back to step 5 (the last correct one)
Generate a new sequence of thoughts from step 6
→ alternative reasoning
The loop repeats until the answer is correct or the iteration limit is reached.
Example
Task: You're advising a healthy-meal subscription startup. You need to work out the unit economics: what it costs to acquire a customer, how much they'll bring in, when they pay back. The numbers: CAC $40, average order $59/month, margin 40%, average LT 8 months, churn 15%/month. Accuracy matters: an investor is asking about ROI.
Prompt:
Task: Calculate the unit economics for a subscription business.
Data:
- CAC (customer acquisition cost): $40
- Average order: $59/month
- Margin: 40%
- Average LT (lifetime): 8 months
- Churn rate: 15%/month
Reason step by step. Pause after each step.
Output structure:
STEP 1: [first thought/calculation]
---
STEP 2: [next thought/calculation]
---
...
Stop after the final answer.
Once you give an answer, I'll check it. If it's wrong, you'll analyze
your steps, find the first error, go back to the last correct step
and continue the reasoning differently.
Result:
The model produces a sequence of numbered steps: margin revenue per customer, multiplied by LT, minus CAC, then the payback period. Each step is a separate block with an explicit boundary.
If the solution has an error (say, it forgot to factor churn into retention), you say "wrong." The model analyzes its steps, finds "STEP 3: error, didn't account for the effect of churn on average LT," rolls back to STEP 2 and builds an alternative continuation: recalculates retention with churn, corrects LT, and produces the right final numbers.
Instead of recalculating everything from scratch, you get a targeted fix of one specific step.
Why it works
An LLM can't reliably localize errors in a continuous stream of text. When a model generates Chain-of-Thought the usual way, you get an unbroken sequence of tokens with no clear boundaries between logical steps. Finding "exactly where the error is" means parsing meaning out of paragraphs of text. The model often misses: it points to a step that already contains the consequences of an earlier error, or can't find it at all.
LLMs are good at generating sequences and following structured instructions. When at every step the model gets the prompt "state the next complete thought," it makes a deliberate decision about that step's content. That creates natural decision boundaries: points where one thought ends and the next begins. The structure is explicit: thought 1, thought 2, thought 3...
The method uses these boundaries as anchors for localization. The model walks through the numbered steps and analyzes each one separately. "Step 1: correct. Step 2: correct. Step 3: there's a calculation error here." The study showed that large models land on a clean prefix (the prefix before the first error) 97% of the time with structured reasoning, versus a much lower rate on continuous CoT. From a clean prefix, resampling works well: the model builds alternative reasoning on a correct foundation.
Control levers:
- Stopping marker between thoughts (
---,[NEXT],</thought>): pick whatever is easiest to parse - Step granularity ("one calculation" vs. "one logical block"): affects localization accuracy
- Maximum iterations of the correction loop: a trade-off between quality and tokens
- Verification source (oracle = you check it yourself / self = the model checks itself): oracle is more accurate, self is more autonomous
Prompt template
Option 1: Structured generation with manual correction
Task: {task_description}
Reason step by step. Each step is one complete thought (observation, calculation, deduction).
Output format:
STEP 1: [thought]
---
STEP 2: [thought]
---
...
STEP N: [final answer]
Pause after each step. Move to the next one only once the current one
is fully stated.
How to use it: After generation you check the answer. If it's wrong → ask the model to analyze its steps and find the error → it returns a step number → you write "go back to step N-1 and continue differently."
Option 2: Autonomous self-correction loop
Task: {task_description}
INSTRUCTIONS:
1. Generate the solution step by step. Number each step.
Format: "STEP X: [thought]"
2. When you're done, check the answer yourself.
3. If the answer is wrong:
- Analyze each step
- Find the first error (logical, computational, conceptual)
- Return: "Error in STEP X: [explanation]"
- Roll back to STEP X-1
- Generate an alternative continuation
4. Check again. If it's still wrong, repeat the analysis and correction.
Maximum of {N} correction iterations.
Start by generating the first version of the solution:
Fill in: {task_description}: your task; {N}: the number of correction attempts (2-3 is enough).
🚀 Quick start: paste into a chat:
Here's the Thought-ICS method for precise self-correction. Adapt it to my task: [your task].
Ask me questions so you can structure it properly.
[paste Option 2 above]
The model will ask about the details of the task, how many iterations, and whether it should verify on its own or you'll verify by hand. It will take the step-by-step generation → localization → rollback pattern and adapt it to your case.
Limitations
⚠️ Simple tasks: For questions like "capital of France" the structure is overkill; regular CoT is enough. The method pays off when the reasoning is complex and the chance of error is high (math, multi-step analysis, calculations).
⚠️ Tokens: The correction loop needs more tokens than a single generation: several reasoning branches + analysis + localization. On hard tasks the accuracy gain justifies it; on easy ones it's wasteful.
⚠️ Self-verification isn't perfect: When the model checks correctness itself (no external verifier), it can miss an error or wrongly reject a correct answer. The study found that V/L disagreement (verification says "wrong," localization finds no error) is a common exit point under self-verification. Oracle verification (you check it yourself, or there's ground truth) is more reliable.
⚠️ Small models: Localization works better the bigger the model. 3B-8B models produce a clean prefix (a clean rollback to before the first error) less often: they frequently miss the real first error and roll back to a prefix that still contains the bug. 70B+ models hit a 97% clean-prefix rate.
Why structure beats continuous CoT
The researchers compared Thought-ICS (structured generation) with Token-ICS (regular CoT + an attempt to localize the error in the text). Both used the same framework: verification → localization → rollback → resampling. The only difference was generation: thought by thought vs. token by token.
They tested eight models (from 3B to 120B parameters) on six benchmarks: math competitions (AMC, AIME), hard MATH-L5 problems, arithmetic word problems (MathQA), common sense (CSQA), PhD-level science (GPQA). 100 problems per dataset.
The key to a fair experiment: they used oracle verification, where an external verifier (ground truth) said "the answer is wrong," to isolate the ability to localize and correct from verification problems. If the model can't tell there's an error at all, correction never kicks in. The oracle removed that barrier.
The finding: Thought-ICS showed a 20-40% lift (accuracy gain over baseline CoT) on almost every model and dataset. Token-ICS gave at best a 5-10% lift, and often no improvement at all, or even made things worse.
Why such a gap? They ran a detailed localization analysis. They took three frontier models (Sonnet-3.7, GPT-4.1, GPT-o1-mini) as an oracle localizer: each independently analyzed the faulty solutions and pointed to the first error. If all three agreed, that was the "ground truth" localization. Agreement on structured reasoning: 51% exact match, 74% within ±1 step, 85% within ±2 steps. On continuous CoT agreement was much lower: the models quoted different fragments of text, and even with a tolerance of 10% of the solution length, agreement was only 58%.
Then they compared self-localization (the model finds the error itself) with the oracle. On structured reasoning, large models (70B+) landed on a clean prefix 97% of the time, rolling back to the point before the first error or earlier. Mid-size models (7-32B) hit 53-71% clean prefix. On continuous CoT even large models often missed.
The critical insight: when the model rolls back to a clean prefix (no errors), resampling works several times better. From a clean foundation, the model builds alternative reasoning with a ~60-80% chance of success. When the prefix is erroneous (it contains an error that was missed), the chance of a successful fix drops to ~20-30%, because the model reaches the wrong conclusion again, inheriting the early error.
Bonus: structured generation by itself (even without correction) gave a small initial lift: the model solved problems correctly more often on the first try. The authors suggest that an explicit pause between thoughts makes the model phrase the next step more deliberately, cutting down on impulsive errors.
They also tested a self-verification mode (no oracle): the model solves, checks, finds the error and corrects on its own. Result: still a net positive lift, though smaller than with the oracle, because self-verification sometimes misses errors or wrongly rejects correct answers. But Thought-ICS still beat existing self-correction methods (Self-Refine, CoVe) that operate on continuous CoT.
Original from the paper
Prompt for step-by-step generation (Thought MDP):
State your next reasoning step (one observation, calculation, or deduction).
The model outputs one thought, then a special closing marker (for example --- or </step>). The system appends that thought to the prefix and gives the same prompt again. The loop runs until the final answer (for example \boxed{...}).
Prompt for error localization:
You generated the following reasoning trace:
[STEP 1] {thought 1}
[STEP 2] {thought 2}
...
[STEP N] {thought N}
Analyze each step carefully. Identify any logical flaws, arithmetic mistakes,
or incorrect assumptions.
If you find an erroneous step, return:
- The step number of the FIRST error
- Brief explanation of what went wrong
If no error is found, state "No error identified."
Context: The researchers used these prompts to test models via API. Step-by-step generation built a reasoning tree: each thought is a node, a rollback is a move to the parent node, resampling is a new branch. At inference time the framework drove the loop: generation → verification → localization → rollback → new generation.
Adaptations and extrapolations
💡 Adaptation: a simplified manual mode
If you don't want to automate the loop, use the structure by hand:
Step 1: Ask the model to solve the problem step by step with explicit numbering.
Solve the problem step by step. Number each step: "STEP 1:", "STEP 2:", etc.
Problem: {your_problem}
Step 2: Check the final answer. If it's wrong → ask for an analysis:
The answer is wrong. Analyze your steps. Find the first error.
Return the step number and an explanation.
Step 3: Once you have the number (e.g., "error in STEP 5") → roll back:
Go back to STEP 4. Continue the solution from that point,
but reason differently at STEP 5.
The value: control at every stage. You see what the model localized, you can agree or push back, and you can pick the rollback step yourself.
🔧 Technique: add an explicit final checklist
After the final answer, ask the model to check itself against criteria:
STEP N: [final answer]
FINAL CHECK:
□ Are all calculations correct?
□ Is the logic free of contradictions?
□ Does the answer match the original question?
□ Are the units correct?
If any item is NO, return the number of the step with the error.
Effect: reduces localization misfires (when the model doesn't see the error) and adds a structured self-check.
🔧 Technique: meta-reflection after correction
After resampling, ask the model to explain what changed:
You rolled back to STEP X and generated a new continuation.
Explain:
1. What was wrong in the first version?
2. What changed in the second version?
3. Why is the new reasoning more correct?
Effect: a transparent correction. You see not just the final answer but the model's understanding of what exactly it fixed. Useful for learning and for complex cases where you need confidence in the conclusions.
Resources
"Structure Enables Effective Self-Localization of Errors in LLMs": the original study, which tested Thought-ICS on eight models (LLaMA 3, Qwen 2.5, GPT-OSS) and six reasoning benchmarks.
The authors cite work on the anterior cingulate cortex (the neuroscience of error monitoring at discrete decision points): Botvinick et al. 2004, Carter et al. 1998, MacDonald et al. 2000, as inspiration for the method's architecture.
Related self-correction methods: Self-Refine (Madaan et al., 2023), CoVe (Dhuliawala et al., 2023), Reflexion (Shinn et al., 2023), RCI (Kim et al., 2023). Thought-ICS differs from them by having explicit structure and pinpoint localization instead of holistic regeneration of the entire solution.
Thought-level generation was used in Tree-of-Thoughts (Yao et al., 2023), but there the segmentation is rule-based and external scoring is required. Thought-ICS lets the model decide the boundaries of its thoughts itself.