TL;DR
The longer your conversation with an LLM, the less text the model can generate in a single reply. It's not a bug and not a hallucination — it's a physical constraint: once the context (the chat history) is half full, the model's ability to produce long answers drops faster than you'd believe. The researchers call this output stalling — the model simply returns an empty response with no error, and you assume something broke.
The main pain: you've built up a long analysis session with Claude or GPT-4o, and now you want it to write a 10-page report — and you get truncated text or nothing at all. Technically, the token limit isn't used up. The problem is that capacity was 100% at the start, and by mid-conversation only 40–50% of the theoretical maximum is left.
The fix is two-step generation: content first, formatting second. Instead of "write a report in a nice format," first ask for structured text with no formatting, then format it in a separate request. This approach saves 48–72% of tokens and completely eliminates empty responses.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1 (one request): Generate the content
Task → "Write the content without formatting,
as structured text" → plain text / JSON
STEP 2 (separate request): Apply the format
Content from step 1 → "Format it as [target format]" → final document
Both steps happen in a regular chat. No code.
Example in Practice
Task: You've had a big session in Claude — analyzing three competitors, picking apart their websites, discussing positioning. Now you want it to write a 6–8 page sales proposal for a client, with sections, headings and conclusions. After the long analysis, the model starts to "choke" — the text gets cut off, the structure falls apart.
Prompt (Step 1 — content):
Based on our competitor analysis, write the content
of a sales proposal for [client name].
Output everything as structured text — no markdown,
no tables, no headings with special characters.
Just plain text, split into meaningful blocks,
with each block's name in square brackets.
Blocks: client's problem, our solution, how we differ from competitors,
case studies, terms, call to action.
Write the full content, nothing abbreviated.
Prompt (Step 2 — formatting, new request):
Take this text and format it as a professional
sales proposal:
- Add headings and subheadings
- Bold the key ideas
- Use bullet lists where appropriate
- Add a closing block with a call to action
[paste the text from step 1]
Result: Step 1 returns the full text of every section — the model can manage it because plain text is cheap in tokens. Step 2 works with short, ready-made content instead of generating from scratch. The final document comes out complete, with nothing cut off.
Why It Works
An LLM doesn't generate "as much as fits" — it has a hidden internal limit. When you start a conversation, the model can produce roughly 40,000 tokens in one reply. But by the time the chat history fills 40–50% of the context, real capacity drops to 20–25%. That's because the model "looks at" the entire history for every word it writes — the more text behind it, the fewer resources for what's ahead.
Formatting costs tokens — more than you'd think. Markdown adds +5% to the text volume. JSON adds +15%. HTML adds +20%. Word-like formats add +40%. When the model writes a big structured document in one shot, it burns a large chunk of its already-shrunken capacity on brackets, tags and markup symbols instead of content.
Splitting the work is a detour through the model's strength: LLMs generate plain text well even with a full context. Formatting ready-made text is even easier and cheaper. Two small steps are more reliable than one big one.
Control Levers
- Formatting level in step 1 — the simpler ("text only, no special characters"), the fewer tokens and the higher the reliability
- Detail in step 1 — explicitly ask for "the full content," otherwise the model will compress
- Number of steps — for very long documents, split step 1 into parts: "write blocks 1–3," then "write blocks 4–6"
- When to start — if you need a long document, it's better to start a new chat than work inside a bloated one. Context at 0% → maximum capacity
Prompt Template
Step 1 — content:
Based on {what was in the conversation / the material}, create
the content of {document type}.
Output format: plain text with no formatting.
Start each section with its name in [square brackets].
Sections: {list of sections}.
Write it in full, no abbreviations and no "..." at the end.
Step 2 — formatting (new request):
Format this text as {document type}:
{formatting requirements}
[paste the text from step 1]
What to fill in:
- {what was in the conversation} — "our analysis," "the uploaded document," "the discussion above"
- {document type} — sales proposal, analytical report, technical spec
- {list of sections} — list the sections you need
- {formatting requirements} — headings, bold, tables, bullets — whatever you need
🚀 Quick start — paste into the chat:
Here is a two-step document generation template.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
Step 1 — content:
Based on {what was in the conversation / the material}, create
the content of {document type}.
Output format: plain text with no formatting.
Start each section with its name in [square brackets].
Sections: {list of sections}.
Write it in full, no abbreviations and no "..." at the end.
---
Step 2 — formatting (new request):
Format this text as {document type}:
{formatting requirements}
[paste the text from step 1]
The LLM will ask what document you need, which sections, and what was in the context — because without that it can't fill in the placeholders or suggest the right structure.
Limitations
⚠️ Short documents: For texts under 1–2 pages, splitting isn't needed. The model will handle it in one request, and two steps only add friction.
⚠️ Markdown documents: Markdown adds only +5% to the text volume — the gain from splitting is minimal. The method works better for structured formats: reports with tables, technical documents, presentations.
⚠️ Long chats without memory: If the context is already overflowing (Claude starts writing a "conversation summary" on its own), even step 1 may not deliver the full text. In that case — a new chat with a short summary of the previous one.
⚠️ Step 2 quality: If step 1 produced overly compressed text, step 2 won't restore the details. Explicitly demand completeness in the first step.
How They Studied It
The team didn't dream this method up in theory — they ran into a real failure. An agent on Claude 3.5 Sonnet analyzed nine research papers (building up ~90,000 tokens of context), was given the task of generating a formatted Word document — and returned an empty response five times in a row, burning ~50,000 tokens for nothing. The model reported no error. Just silence.
To understand why, the researchers measured how real generation capacity changes for three models (Claude 3.5 Sonnet, GPT-4o, Llama 3.1 70B) depending on how full the context is. The result was surprising: all three models lose half their capacity at just 55% context fill, and the degradation is nonlinear — it accelerates as the context fills. It doesn't depend on the size of the context window: a model with 200K tokens shows the same effect as one with 128K — the "cliff" just shifts.
Then they tested three strategies on four document types, five runs each. Direct generation (ask for the finished document right away) failed in nearly every case with a full context: 0/5 successes for Claude and GPT-4o. Chunking (split into parts) — 60% success, but more expensive in tokens. Two-step (content → format) — 5/5 for all three models, 60 out of 60 runs, while saving 48–72% of tokens. Interestingly, the two-step approach isn't just more reliable but also faster — 2.5x faster on the report example. The model doesn't "think longer," it just doesn't stall.
Original from the Paper
The paper has no ready-made prompt — it's a theoretical framework with a software implementation (Gen-Pilot). The main practical principle is expressed through the Decision Cascade:
s* =
direct if T_direct ≤ OGC_eff · β
chunk_k if T_direct/k ≤ OGC_eff · β
defer otherwise
where β = 0.67 (safety margin — use only 67% of the estimated capacity)
What this means in practice: if the expected document length is small → generate directly. If it's large but can be split → split it into parts. If nothing fits → use the two-step approach (content separately, formatting separately).
Adaptations and Extrapolations
💡 Adaptation: The 70% Rule for Long Tasks
The researchers use a safety margin of β = 0.67 — don't push the model to its limit. Apply this rule deliberately: if the conversation is already long, don't ask for "write everything at once." Ask for 70% of the volume in the first request, check the quality, then ask for the rest.
🔧 Technique: Explicitly demand completeness → prevents truncation
Add this to step 1: "Write it in full. Don't abbreviate, don't use '...' or '[to be continued]'. If it doesn't fit, say so — don't cut off." This forces the model to choose: deliver the full text or honestly say it can't — instead of silently truncating.
💡 Adaptation: A new chat for long documents as basic hygiene
The simplest takeaway from the study: if you need a long document, start a new chat. Carry over only the key data, not the whole conversation history. Context at 0% = maximum generation capacity. It's cheap and it always works.
🔧 Technique: Format-light input
When you hand a large text to the model for analysis, pass it without extra formatting. Strip complex markdown before pasting: fewer tokens in → more capacity out.
Resources
Title: "When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis"
Authors: Justice Owusu Agyemang, Michael Agyare, Miriam Kobbinah, Nathaniel Agbugblah, Prosper Addo
Affiliations: Sperix Labs, VIA Cybersecurity Lab (KNUST, Ghana), GCTU, NCA
Implementation: Gen-Pilot (open-source MCP server) — [email protected]
Date: April 2026