TL;DR
Thinking Inertia shows that neither the instruction “answer right away, don't think” nor a disabled thinking mode removes reasoning from the answer. The authors split the response into two parts: the text before the final answer and the answer itself. They check whether the first part contains at least one inference step and measure how the wording of the question affects this.
The main finding: the amount of reasoning is set not by the switch but by the shape of the question. On yes/no questions, models without thinking answer cleanly 80–100% of the time. On multiple-choice questions the results vary. On open-ended questions the model still works something out along the way. For DeepSeek-V4-Flash with thinking mode off, visible inference remained in 99.9% of open-ended answers. A strict ban on reasoning raises the share of “bare” answers from nearly zero to about 40%, but accuracy drops by 15 percentage points on average.
The practical takeaway is simple: if you need a short answer with no explanation, don't coax the model into “not thinking” — change the shape of the question. Give it ready-made answer options, and a clean answer becomes much easier to get. For open-ended tasks, banning reasoning hurts accuracy.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
This is a finding, not a step-by-step technique. Below is the decision flow that follows from it.
QUESTION: do you need a short answer with no explanation?
│
├─ Can it be reduced to yes/no → ask a closed question
│ → you get a clean answer almost every time
│
├─ Can you give a list of options → list the options in the request itself
│ → you get a clean answer noticeably more often
│
└─ The answer is open-ended (calculation, text, solution) →
DON'T ban reasoning hard
→ otherwise accuracy drops; strip the explanations afterwards
Everything happens in a single prompt. No separate requests needed.
Example in Practice
Task: You're building an automation in n8n or Make for an Amazon seller support desk. An email goes into the model, and the output must be a single label: “returns”, “logistics”, “fees”, “listing suspended” or “other”. The next block in the workflow picks up the label. If the model writes “Let's look at this email… the customer is complaining about…”, the block breaks. “Answer without reasoning” doesn't always do the trick.
Prompt:
You classify emails from marketplace sellers.
Email:
“Hi! This is my third time writing: the goods were shipped to the Dallas warehouse 9 days ago,
the status in Seller Central isn't changing, the shipment is stuck. When will it be received?”
Pick exactly one category from the list:
A) returns
B) logistics and shipments
C) fees and payouts
D) listing suspended
E) other
Answer format: a single letter only (A, B, C, D or E). Nothing before or after it.
Result: The answer comes back as one letter that's easy to parse automatically. A list of options in the request makes a clean answer more realistic than asking “answer in one word” on the open question “what topic is this email about?”. For critical workflows, spot-check whether lead-in phrases appear before the letter.
Why It Works
The LLM's weakness. The model is used to talking its way to the answer. If the question is open-ended, it has nothing to replace that path with. It has to “generate” the answer itself, and the rough inference goes straight into the text. That's why “don't think” works poorly: the instruction says what not to do, but gives the model nothing to lean on instead of reasoning.
What the model does well. If the answer already sits in the request — a yes/no or a list of options — the model only needs to pick. Picking doesn't require intermediate steps, so clean answers come out often. The authors tested this directly: they rewrote the same questions in different formats, and reasoning-free answers were easiest to get in formats with visible candidates.
How to use it. The lever isn't the “don't think” instruction, it's the shape of the question: - A closed list of options → shortens the path to the answer. - A strict output format → tells the model what to write instead of lead-in words. - An open-ended task → don't push with a ban, or you'll lose accuracy. In the paper this cost about 15 pp.
Prompt Template
{context_or_text}
Question: {question_in_closed_form}
Options:
A) {option_1}
B) {option_2}
C) {option_3}
D) other
Answer format: only {letter_or_word}. Nothing before or after the answer.
What to fill in:
- {question_in_closed_form} — rephrase “why/what/how” as “which of the options” or “yes or no”.
- Options — the real categories of your task. Always keep “other” so the model doesn't force-fit an answer.
- If there are no options and the task is open-ended, this template doesn't fit. Let the model think, and clean up the answer in a separate step.
Limitations
⚠️ Banning reasoning hurts accuracy: on open-ended tasks (in the paper — school and competition math), a strict “answer immediately” raised the share of clean answers only to ~40%, while accuracy dropped by ~15 percentage points on average.
⚠️ The switch doesn't guarantee the result: a disabled thinking mode and the absence of
<think>tags don't mean there's no reasoning. It just moves into the regular answer text.
⚠️ A short answer isn't the same as an answer without inference: a compact one-line formula already counts as a reasoning step.
⚠️ Study conditions: they tested benchmark questions with verifiable answers (yes/no, multiple choice, math). How this behaves on long business texts and in agentic scenarios isn't shown in the available portion of the paper. The authors didn't study hidden reasoning inside the model.
⚠️ Partial text: this breakdown relies on the introduction, methodology and the start of the results. Detailed per-model tables and appendices were unavailable.
How They Studied It
The authors started from the fact that the very notion of “the model isn't thinking” was fuzzy. It was defined by indirect signs: the mode is off, there are no <think> tags, the answer is short. They showed all three signs are easy to fool. So they split the response into “text before the final answer” and “the answer itself” and measured three things. First — whether the response consists strictly of the answer only. Second — how related the text before the answer is to the question. Third — whether it contains an explicit inference step (judged by an LLM judge).
They tested six models: three sizes of Qwen3, DeepSeek-V4-Flash, GPT-5.4 and Gemini-3-Flash. They used six benchmarks in three formats: yes/no (BoolQ, StrategyQA), multiple choice (MMLU, MMLU-Pro) and open-ended problems (GSM8K, MATH). The models were tested in six modes: from thinking enabled to a “don't think” ban and a forced answer prefix (“The answer is \boxed{”).
To trust the metrics, the team validated them with humans: 600 answers, five annotators, 98.8% agreement with the LLM judge. They also brought in a second judge from a different model family (Claude Opus 4.8) so the result wouldn't hinge on one model. The same question was rewritten into different formats to rule out the “the task is just harder” explanation.
The surprise: a disabled thinking mode changes almost nothing on open-ended questions. The practical takeaway: steer with the shape of the question, not the instruction.
Original from the Paper
Answer directly without thinking
Think step by step
please answer <Question> directly
The answer is \boxed{
Context: These are fragments from the mode descriptions. “Answer directly without thinking” is an example of a strict ban, “Think step by step” is the CoT mode, and “The answer is \boxed{” is the forced answer prefix (PREFIX) that's inserted at the start of generation instead of an instruction. The full texts of the SOFT and STRICT modes from Table 2 aren't given in the available portion.
Adaptations and Extrapolations
💡 Adaptation for coding with an agent: same principle, but for quick checks in Claude Code or Cursor. Instead of “answer briefly, don't reason”, give a closed choice.
Can this PR be merged without further changes? A) yes B) no, there are blocking comments C) needs a check by the author Answer with one letter.
🔧 Technique: separate the reasoning from the answer (my extrapolation, not tested in the paper). If the task is open-ended (a calculation, unit economics) and you need parseable output, don't ban reasoning. Give a format: first a “Reasoning” block, then an “ANSWER: ...” line, and grab only the last line. That way you don't lose accuracy and still get a clean result.
Resources
- Thinking Inertia: LLMs Keep Thinking When Told Not To
- Website: https://thinking-inertia.github.io/
- Code: https://github.com/thinking-inertia/code
- Data: https://huggingface.co/datasets/thinking-inertia/llm-nothinking-trajectories
- Authors: Dianqiao Lei (Tsinghua University), Kevin Qinghong Lin, Philip Torr (University of Oxford), Pan Lu, James Zou (Stanford University)