Chain-of-thought prompting: what the research shows

What chain-of-thought prompting is

Chain-of-thought (CoT) prompting asks the model to write its intermediate reasoning before it commits to an answer. The simplest form is a single phrase such as Think step by step. The stronger forms replace that phrase with a fixed route: pull out the facts, match them against the criteria, explain the logic, and only then answer.

The research on this site splits CoT into two families:

• Free-form reasoning. The model decides what to write. This is the classic zero-shot trigger, and on hard professional material it can be the difference between guessing and working through the problem, as the CFA Level III essay study describes.
• Templated reasoning. You define the steps or slots and the model fills them. In a comparison of four prompt styles for text classification, a step-by-step decomposition (extract evidence, match to category definitions, explain, answer) beat plain instructions and few-shot examples: accuracy rose from about 0.55 to 0.775 and agreement (kappa) from about 0.45 to 0.744.

Most of the useful work in CoT happens in that second family: the template decides what the model is allowed to think about.

Why it works

A language model writes one token at a time, and every new token is conditioned on everything already on the page. Reasoning written before the answer becomes part of that context, so the answer is built on stated intermediate conclusions rather than on a quick pattern match to the question.

The same mechanism explains why order matters as much as the presence of reasoning. In a production LLM-judge pipeline, six rounds of rubric rewrites barely moved the judge, while simply putting the reasoning field before the score field raised agreement with human experts from 43% to 52%. When the score comes first, the explanation is written to defend a number that already exists.

Written reasoning also switches which knowledge the model draws on. An implicit-bias study across 15 stereotypes found that turning on reasoning cut word-association bias from 0.41 to 0.13 in GPT and from 0.59 to 0.18 in Claude, while Gemini and Llama barely moved. The authors read this as reasoning activating the values learned in alignment, which only works where such values exist.

A scratchpad prompt adds a practical reading of the same idea: a separate space for working notes acts as external memory, so the model does not have to juggle conditions and output at once.

Variants that tested well

The variants that held up in testing all narrow the reasoning rather than lengthen it.

• Slot templates for code. A study of five CoT paradigms for code generation found that structured CoT (problem analysis, algorithm, implementation) reached 85-95% of the accuracy of long reasoning-model traces at roughly 10% of the tokens, while plain Let's think step by step often made code worse. The gain was +12% for GPT-3.5 and only +1.5% for GPT-5.
• Facts first, then reasoning. Focused CoT first extracts the problem into numbered tags, then reasons only over that block. On Qwen-3 models from 0.6B to 32B it used 2-3x fewer tokens at comparable accuracy.
• An explicit decomposition plan. Writing out the subtasks and their order helped even reasoning models in a Text2SQL benchmark of six workflows: o4-mini went from 56.27% to 65.53% with decomposition plus three worked examples.
• Linked-question steps. Derivation prompting forces the model to state what changed in the input and what must change in the output before answering a follow-up question (+15.2% accuracy over plain CoT and few-shot in the authors' tests).
• Formalise, then solve. Formula-first prompting writes givens, unknowns and equations before choosing how to compute (+13.3% on finance problems over plain CoT). A diagram-based variant replaces prose with a Mermaid graph, and a timeline variant sorts events by date before asking chronological questions.
• Cross-check two kinds of reasoning. CoT-PoT solves once in words and once in code; when the two agree, the answer is accepted, which settled 78.6% of problems in two calls.

When it helps and when it hurts

CoT is reliable on problems with a checkable answer: arithmetic, logic, multi-step analysis, classification against explicit criteria. Outside that zone the evidence turns negative quickly, and the adaptive reasoning page covers how to decide when to switch it off. The main failure modes:

• Compression tasks. Reasoning models wrote wordier, less accurate dialogue summaries than GPT-4o, and in a comparison of 8 strategies on 8 summarization datasets a plain prompt with examples often matched the elaborate ones.
• Reading people. On theory-of-mind benchmarks, raising o3's reasoning effort dropped accuracy on hard items from 0.838 to 0.693.
• Talking to users. Forced thinking made customer-support agents share less information; Gemini-2.5-Pro lost 37% on retail tasks.
• Holding a constraint. CoT pulled forbidden knowledge into the answer on all three models in a knowledge-cutoff study, and on decisions with a biased anchor it built a more convincing case for the biased option. Stating the principles first cut bias by about 51%.
• Strong models and strict scaffolds. A detailed rule set helped GPT-4o (97% vs 93%) but hurt GPT-5 (94% vs 96.4%) in a GSM8K test of prompt complexity, and across 23 techniques on 12 models stronger models did best with simple prompts.

One more caution: reasoning that is cut off mid-way can be worse than none. Under a tight token budget, DeepSeek scored 17% with truncated CoT and 53% with a direct answer.

{task_description}
Use this structure:
**Problem analysis:**
- What needs to be solved?
- What are the inputs/outputs?
- What are the edge cases?
**Algorithm:**
- Which steps to perform?
- Which data structures to use?
- Which checks are needed?
**Implementation:**
[code with comments]
{additional_requirements}

Use ONLY the facts from <context> below. Do not refer to the original problem.
Instructions:
1) Reason step by step
2) When you use a fact, cite <info_k>
3) Final answer in \boxed{}
{context_from_step_1}

Here's the ad post text: {text}.
First, break it down in detail:
- Are there any stretched arguments or exaggerations?
- Are there reputational risks if the post lands differently than intended?
- Is the logic actually convincing, or does the text just sound smooth?
Don't praise nice phrasing — look for substantive problems,
even if the text reads easily.
Don't mention a score until you've finished the breakdown.
Only at the very end, give a final score from 1 to 10.