Context Rot: Why a Model Starts Giving Up in Long Sessions, and How to Stop It

Two independent approaches you can use together:

APPROACH A — Context management (within a session)

STEP 1: Run a long conversation → watch for rot signals
         (the model writes "I can't determine," "possibly," "there isn't enough data")
STEP 2: A trigger is hit (N messages OR signals appear) →
         ask the model to compress the context into a summary
STEP 3: Start a new chat with the summary instead of the full history →
         keep going without the "tail" of failures

All steps are done by hand in a regular chat.

APPROACH B — Filtering results (parallel attempts)

STEP 1: Ask the same hard question 3-5 times in separate chats
STEP 2: Look at the type of answer: confident ("here's the answer") vs. gave up ("can't find it")
STEP 3: Keep only the confident answers → compare them against each other

Each request is a separate chat, so there's no cross-contamination.

Task: You're doing deep research for a new project, looking for an untapped niche in the US online education market. You're already 40 messages in with Claude. You've started noticing the model writing more and more often "it's hard to say with confidence," "there isn't enough data for a definitive conclusion," "it may be worth checking this yourself."

That's textbook context rot. The model has piled a heap of dead ends from the earlier search into the context, and now it generates with all that uncertainty baked in.

Context-compression prompt (run it every 10-15 messages):

Stop. Before we continue, write a progress summary.

Summary structure:
1. Research goal (one sentence)
2. What we've DEFINITELY established (only high-confidence facts)
3. What we checked and ruled out (dead ends, briefly)
4. What hasn't been checked yet (open questions)
5. The next logical step

Only what you're sure of. No "possibly" or "probably" in items 1-2.

Then open a new chat, paste the summary in as the system context, and continue the research without the dead weight of failures.

Result: The model produces a structured summary of 200-400 words. In the new chat it starts reasoning confidently, without the "tail" of 40 messages full of failed attempts. Answer quality improves noticeably: more specifics, fewer caveats.

An LLM generates the next token based on everything that came before it. If the history is full of patterns of failed attempts ("searched, didn't find it," "checked, doesn't fit," "not much data"), those patterns weigh on generation. The model sort of "catches" the tone of the context and starts reproducing it: it gives up or hedges.

LLMs are good at one thing: continuing an established pattern. If the context is the story of confident, methodical research, the model keeps going in the same spirit. If the context is a dump of attempts and failures, it reproduces the uncertainty.

Compressing the context removes the failure pattern and leaves only progress. The model looks at a clean summary of achievements and keeps working productively. Not because it "remembers less," but because the pattern it relies on is now a healthy one.

Control knobs: - Compression frequency → every 10 messages is expensive, every 20 is a compromise. One compression at the end beats none - Filter strictness → "only high-confidence facts" in the prompt strips out junk more aggressively than just "takeaways" - Number of parallel attempts → 3 is enough for simple tasks, 5-8 for critical decisions - Rot signals → marker words: "possibly," "hard to say," "not enough data," "can't determine" serve as triggers for compression

Template A — Periodic context compression:

Stop. Write a summary of our progress on the task: {task}.

1. What's definitely established (confident conclusions only)
2. What we checked and ruled out
3. What remains open
4. The next priority step

No "possibly" or "most likely" in items 1-2.
If you're not sure, put it in item 3, not item 1.

Template B — Rot-aware filter for parallel attempts:

Rate this answer on one criterion: did the model answer CONFIDENTLY or did it GIVE UP?

Confident answer: gives a concrete result; the reasoning leads to a conclusion
Gave-up answer: "I can't determine," "there isn't enough data," an answer full of constant caveats

Answer to rate:
{paste the model's answer}

Output: CONFIDENT / GAVE UP + one sentence explaining why.

Placeholders: - {task} — what you're researching/writing/analyzing - {paste the model's answer} — copy the answer you want to check for rot

🚀 Quick start — paste into a chat:

Here's a template for fighting context degradation in long sessions. 
Adapt it to my task: {your task}.
Ask me questions to set it up properly.

[paste the template above]

The LLM will ask about compression frequency and the type of task, because the trigger for a "stop and summarize" depends on how fast junk piles up in your particular workflow.

⚠️ Simple tasks: You don't need the method for short conversations (under 10-15 messages). Rot sets in during long research sessions.

⚠️ Context isolation (sub-agents): Delegating tasks to separate "specialists" (separate chats) depends heavily on the model; with weaker models it can make results worse. It only works well with top-tier models.

⚠️ Compression kills details: The model may leave out important details from the summary that turn out to matter later. If the task requires reproducing every step exactly, write the summary yourself; don't trust the model with it.

⚠️ Parallel attempts are token-hungry: The "5 times with filtering" approach multiplies your spend by 5. For quick requests, it's overkill.

Title: Diagnosing and Mitigating Context Rot in Long-horizon Search (2025)

Authors: Shijie Xia, Yikun Wang, Zhen Huang, Pengfei Liu

Organizations: Shanghai Jiao Tong University, Fudan University, SII, GAIR

Code: https://github.com/GAIR-NLP/ContextRot

Related concepts: ReAct framework (Yao et al., 2023), FoldAgent (Sun et al., 2025)