TL;DR
The researchers tested how different reasoning formats hold up under a token limit. They took 4 top models (GPT-5.1, Gemini, DeepSeek-V3.2, Grok) and had them solve math problems through code, through text explanations, through both formats, or with no explicit reasoning at all. Then they cut the token budget to 10-70% of the optimum and watched what happened.
Something strange turned up: incomplete reasoning can hurt more than no reasoning at all. DeepSeek with a truncated Chain-of-Thought at 50% of the budget scores 17% accuracy — with no reasoning, 53%. Gemini with CoT cut to 10% of the tokens: 17% accuracy; with no reasoning: 67%. It's as if the model gets stuck in a contradictory intermediate state: it declared the variables but didn't finish the logic, started reasoning but stopped mid-sentence — and from that "dangling" position it guesses the answer.
The reasoning format matters more than the token count. Code degrades gracefully — Gemini with code holds 43-47% when cut to 30-50% of the tokens, while text explanations drop to 0%. The hybrid format (code + text) is worse than either pure one — switching between formats burns tokens for nothing. Robustness depends on the model — Grok holds 80-90% at a 30% budget, where OpenAI and DeepSeek fall to 7-27%.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Key Findings
1. Incomplete reasoning is worse than no reasoning
What happens: Under a token limit, a model can produce a better result with no reasoning at all than with a truncated Chain-of-Thought.
The numbers: - DeepSeek-V3.2 at 50% of tokens: no reasoning — 53%, truncated CoT — 17% - Gemini at 10% of tokens: no reasoning — 67%, truncated CoT — 17% - OpenAI at 30% of tokens: no reasoning — 33%, truncated CoT — 10%
Why: Truncated reasoning leaves the model in an inconsistent intermediate state. Variables are declared but never filled in. Logical premises are set up but never completed. When the model is forced to answer from that state, it guesses, and the guess contradicts the logic it had started. A direct answer with no reasoning sidesteps this trap — the model answers right away without getting tangled in contradictions.
2. Code degrades gracefully, text collapses
What happens: When tokens are cut, code keeps working, while text explanations fall apart.
The numbers: - Gemini at 30% of tokens: code — 43%, text — 0% - Gemini at 50% of tokens: code — 47%, text — 0%
Why:
Code has structural redundancy. Even a truncated for loop shows that iteration is happening. An unfinished if statement still signals branching. Every construct carries meaning on its own. Text works differently — it builds long chains of dependencies. Cutting off the phrase "the answer, therefore..." tells you nothing about what comes next. The logical connectors get lost and the reasoning falls apart.
3. The hybrid format is worse than pure ones
What happens: Mixing code and text in one line of reasoning lowers accuracy compared with pure code or pure text.
The numbers: - Grok at 30% of tokens: code — 90%, text — 90%, code+text — 80% - Across all models and all budgets, the hybrid never once beats a single format
Why: Switching between formats creates coordination overhead. The model has to: (1) translate the computational state from code into text, (2) summarize that state in words, (3) encode it back for the next chunk of code. These translations eat tokens without moving toward the solution. A single format spends every token on forward progress.
4. Robustness depends on the model
What happens: Models handle token cuts very differently. Grok holds up; OpenAI and DeepSeek collapse.
The numbers: - Grok at 30% of tokens: 80-90% accuracy - OpenAI at 30% of tokens: 10-27% accuracy - DeepSeek at 30% of tokens: 7-47% accuracy
Notable: On an extreme 10% budget, Grok scores 77% with code and 83% with CoT. Under the same conditions, OpenAI and DeepSeek drop almost to zero.
What to Use in Practice
Principle 1: Short answer → no reasoning
When you're under a token limit (API cap, need a fast answer, saving money):
❌ Don't:
Solve this problem step by step, but keep it short with no extra words.
✅ Do:
Give a direct answer with no explanation.
Why: Truncated reasoning is worse than no reasoning. Better to get the answer straight away than half-baked logic.
Principle 2: Need reasoning + have a limit → use code
When you need reasoning but tokens are limited:
❌ Don't:
Explain the solution step by step with reasoning in natural language.
✅ Do:
Solve the problem with code, no comments. Show only executable code.
Why: Code degrades gracefully — even truncated, it keeps its structure and meaning. Text collapses.
Example:
The task: You need to quickly calculate the ROI of an ad campaign, but your API has a token limit.
Prompt:
Calculate the ROI: we spent $15,000 and got $52,000 in revenue.
Solve it with code, no comments, executable code only.
Result: The model returns compact code with the calculation. Even if it gets cut off, the core logic ((revenue - cost) / cost) stays readable.
Principle 3: Don't mix formats
When you need reasoning:
❌ Don't:
Solve the problem. Use code for the calculations and explain every step in text.
✅ Do:
Solve the problem using code only.
OR
Solve the problem with step-by-step explanations in text.
Why: Switching between formats burns tokens on coordination. A single format is more efficient.
Principle 4: For full reasoning, remove the limits
When the task is hard and you need maximum accuracy:
✅ Do:
Solve the problem step by step. Show all your reasoning. No limit on answer length.
Why: Reasoning models like DeepSeek-V3.2 are trained on long traces. Truncation makes them worse than no reasoning at all. Give them room to reason all the way through.
Why It Works
An LLM generates text sequentially — each new token depends on the ones before it. When you turn on Chain-of-Thought, the model builds an explicit logical chain: "if A, then B, therefore C, so the answer is D." That chain only works as a whole. Cutting it in the middle leaves the model with variables declared but not filled in, and conclusions started but not finished.
From that "dangling" state, the model is forced to guess the answer, and the guess contradicts the logic it had started — like a program reading a variable it never got around to initializing. A direct answer with no reasoning sidesteps the trap — the model generates the final value right away, without getting stuck in intermediate dependencies.
Code degrades gracefully because it has structural redundancy. Each construct (for, if, while, =) carries meaning on its own. A truncated loop still shows iteration. Text works differently — long dependency chains. The phrase "the answer, therefore..." with nothing after it carries no information. Logical connectors ("because," "so," "it follows that") are critical for understanding, and losing them brings down the whole line of reasoning.
The hybrid format spends tokens on coordination — the model translates state from code to text and back, summarizes, re-encodes. None of that moves the solution forward. A single format spends every token on progress.
Limitations
⚠️ Math-only focus: The study tested only math problems (GSM8K, AIME, HMMT). Whether it applies to other domains (text analysis, creative work, structuring) is unknown.
⚠️ Fixed cut percentages: The researchers used 10%, 30%, 50%, and 70% of the optimum. Adaptive budgets (different limits for different tasks) weren't tested.
⚠️ Reasoning models may be especially vulnerable: DeepSeek-V3.2 and o1 are trained on long traces. If a model depends on complete reasoning, it breaks harder when cut short. Regular models may be more robust.
How They Studied It
The team took 4 top models (GPT-5.1, Gemini 3 Flash, DeepSeek-V3.2, Grok 4.1) and 3 math benchmarks of varying difficulty: GSM8K (grade-school math), AIME (olympiad), HMMT (the Harvard-MIT tournament). The idea was simple: make the models reason in different formats, then cut the tokens and see what breaks.
Reasoning formats: 1. Code only — the entire solution as executable code 2. Comments only — reasoning in text form, no code 3. Code + comments — hybrid 4. No reasoning — straight to the answer 5. Regular CoT — no format restrictions
Each model was first run with no limits on a problem to measure how many tokens it used (_Topt_). Then the budget was cut to 10%, 30%, 50%, 70% of _Topt_ and they counted how many problems the model solved correctly.
The surprise: DeepSeek-V3.2 — a model specifically trained for long reasoning — showed the strongest paradox. At 50% of the budget: no reasoning 53%, truncated CoT 17%. The model built for reasoning breaks the hardest from incomplete reasoning. Which shows that depending on complete traces isn't a feature — it's a vulnerability.
Why the results look like this: Reasoning models like o1 and DeepSeek-V3.2 are trained with reinforcement learning on complete reasoning traces. The model is rewarded for the right answer at the end of the full chain. If the chain breaks off, the model lands in a state it was never trained for — hence the degradation. Grok, which held up best, is apparently more robust to incomplete contexts — possibly due to architectural differences or a different training approach.
Practical takeaway: If you work with reasoning models (o1, DeepSeek-R1) under a token limit, it's better to drop reasoning entirely than to make the model truncate its CoT. To save tokens, "give a direct answer" beats "think briefly".
Resources
Broken Chains: The Cost of Incomplete Reasoning in LLMs Ian Su, Gaurav Purushothaman, Jey Narayan, Ruhika Goel, Kevin Zhu, Sunishchal Dev, Yash More, Maheep Chaudhary