TL;DR
The longer a chat runs, the worse an LLM handles the task — not because it runs out of memory, but because old context gets in the way more than it helps. The researchers call this context rot: old exchanges take up room in the model's head and compete with current information, creating noise. And it turned out that trimming the history isn't enough — you need to trim + briefly summarize what you cut.
The study tested four history-management strategies on complex multi-step tasks. The main finding is counterintuitive: the full conversation history gave a worse result (71%) than a trimmed history with a short summary of older messages (91.6%). On top of that, it used 2.7 times fewer tokens and ran 2.5 times faster.
The method works on the principle of "the last N interactions in full + a compressed summary of everything else." Fresh context is needed to work precisely with the current state, and the summary keeps the model from forgetting what's already been done and where things are headed.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
(One cycle — one session. Applied manually in the chat.)
STEP 1: Run the chat as usual
→ History builds up: decisions, attempts, edits
STEP 2: Every 10-15 messages — take a "snapshot"
→ Prompt: "Compress the history: what we decided, what's done, where we're stuck"
→ Output: a compact summary (3-7 sentences)
STEP 3: Open a new chat
→ Paste: [summary] + [your last 4-6 messages + the replies]
→ Keep working with a clean, relevant history
Key principle: Summary → at the start of the new chat. Last 4-6 exchanges → after the summary. The old full history → does not carry over.
Example in Practice
Task: You're helping Mark Davis, the founder of a design studio, nail down the editorial policy for a new media outlet: tone, format, sections, rules. The conversation is already 40 messages long — you've discussed the design, argued about headlines several times and made 5-6 key decisions. The model is starting to contradict itself and circle back to ideas you already dropped.
Prompt to create a snapshot:
You are an editor wrapping up the results of a working session.
Read our conversation above and create a compact snapshot with this structure:
## DECISIONS MADE
[What has been finally agreed — briefly, as bullet points]
## CURRENT TASK
[What we're working on right now — 1-2 sentences]
## OPEN QUESTIONS
[What hasn't been decided yet or is still disputed]
## CONTEXT
[Key constraints, agreements, principles — whatever
is important to remember going forward]
150 words max. Facts only, no fluff.
Then — a new chat:
[Paste the snapshot above]
---
We're continuing work on the editorial policy.
[Paste the last 4-6 messages from the old chat]
Next step: [your task]
Result: The model gets a structured snapshot of the decisions made plus the 4-6 freshest exchanges. The answers get sharper: no more circling back to rejected options, no contradictions with what was already agreed, no "fuzzy" grasp of the current context.
Why It Works
The problem: An LLM treats all tokens in its context roughly the same — old and new have equal "rights." When a chat grows to 40+ messages, early exchanges ("Maybe we add a design section?", "No, let's skip it") keep influencing the answers just as much as the latest ones. That's context rot: the model gets "tangled up" in outdated states and rejected options.
What LLMs are good at: Compression and synthesis — the model can pull the essence out of a large volume of text and keep only what matters. That's its strong suit.
How the method uses it: We ask the model to create a "compressed snapshot" of the history itself — strip out the noise, keep the facts. Then we give it only that snapshot + the freshest context. The result is a precise, lean working memory: orientation in the task (the summary) + the current state (the latest messages).
Control knobs:
| Parameter | Default value | How to adjust |
|---|---|---|
| Snapshot frequency | Every 10-15 messages | Short tasks — less often; iterative ones (edits, development) — more often |
| Summary depth | 150 words | Increase for complex tasks with lots of constraints |
| Number of old messages | 4-6 pairs | Decrease for speed; increase if the task depends on tone/style |
| Snapshot format | Structured (sections) | For simple tasks — a single paragraph of prose |
Prompt Template
Step 1 — Create the snapshot
Read our conversation above and create a working snapshot.
## DECISIONS MADE
[What has been finally agreed]
## CURRENT TASK
[What we're working on]
## OPEN QUESTIONS
[What hasn't been decided yet]
## IMPORTANT CONTEXT
[Constraints, principles, agreements — everything worth remembering]
{word_count} words max. Specifics only.
Step 2 — Start a new chat
## Context from previous work
{snapshot_from_step_1}
---
## Latest messages
{last_4_6_exchanges}
---
## Let's continue
{your_next_task}
Placeholders:
- {word_count} — 100-200 for regular tasks, up to 300 for complex projects
- {snapshot_from_step_1} — paste the output of the first prompt
- {last_4_6_exchanges} — copy the last few "you → model" pairs from the old chat
- {your_next_task} — what we do next
🚀 Quick start — paste into the chat:
Here is a template for creating a working snapshot of a conversation.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about the structure of the task and what needs to be locked in — because the snapshot has to capture exactly what's critical for continuing your specific work, not a generic checklist.
Limitations
⚠️ Linear tasks: If the conversation is short (under 10-15 messages) and not iterative, you don't need the method. The snapshot adds an extra step with no payoff.
⚠️ Tasks that depend on style and tone: If it matters how exactly the previous answers sounded (mimicking a voice, keeping a style), the summary may lose nuance. Carry over the latest messages in full, not just the summary.
⚠️ Details lost in the summary: A short snapshot inevitably loses something. If the task requires every detail to be reproducible (legal wording, exact figures), add to the snapshot prompt: "Numbers, dates and exact wording — verbatim."
⚠️ Studied on automated agents: The research was done on multi-step agents making dozens of tool calls, not on a manual chat. The principles carry over, but the optimal parameters (N=5 for their task) need tuning for your own context.
Resources
Title: Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents
Authors: Abhilasha Lodha, Mahsa Pahlavikhah Varnosfaderani, Abir Chakraborty, Abhinav Mithal
Organization: Microsoft
Key references from the paper: ReAct (Yao et al.), LLMLingua, MemoryBank, ACON, Model Context Protocol (MCP), Microsoft Dynamics 365 Finance and Operations