TL;DR
Context rots. When a conversation with an AI drags on, old faulty reasoning, abandoned hypotheses and irrelevant searches pile up in the model's memory and drag it backward. The model starts retracing paths that were already ruled out and answers worse, even if the right answer was found earlier. This phenomenon is called "context rot."
The key finding: you should compress context based on meaning, not on a schedule ("every 10 messages"). Ask a model to summarize a conversation in the middle of a line of reasoning and it will wipe out exactly what it hasn't used yet. The result is worse than not compressing at all. Timing matters more than frequency.
SELFCOMPACT solves this with a rubric: a short list of conditions the model uses to decide on its own whether to "compress now" or "keep going." The rubric sets two "compress" triggers (a subtask is finished, the conversation is converging on an answer) and two "keep going" vetoes (I'm in the middle of a chain of reasoning, I'm stuck and going in circles). Without a rubric the tool behaves chaotically: models either compress far too often or never use the tool at all.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
Every N steps (tokens / tool calls):
STEP 1: RUBRIC CHECK
→ The model receives the rubric and returns a verdict: COMPRESS or CONTINUE
STEP 2a (if COMPRESS):
→ The model writes a summary of the whole history
→ The history is replaced by the summary, work continues
STEP 2b (if CONTINUE):
→ The rubric and the verdict are removed from the context
→ Work continues unchanged
Both steps happen in one chat, with one model, no external tools
Example in Practice
The task: Alex is launching a B2B SaaS product (task management for small businesses). He has spent an hour-long session with Claude: researching competitors, analyzing pricing, debating positioning, and the conversation is already 40 messages deep. He needs to keep going, but the answers have gotten worse: the model keeps circling back to options that were already rejected.
Prompt:
Evaluate our conversation against the rubric and decide whether we should make a summary now.
COMPRESS if at least one condition is met:
— We've finished a distinct block (competitor analysis / pricing /
positioning) and locked in a clear conclusion
— We're already converging on a final decision and only details remain
CONTINUE if:
— We're in the middle of a line of reasoning and the thought isn't finished
— We're stuck and going in circles — compression won't help
Answer with one word: COMPRESS or CONTINUE.
If COMPRESS — write a summary in this format:
✅ What's decided (facts, conclusions, numbers — exactly)
🔄 What we're exploring now (the current question)
❓ What's still open (next steps)
After the summary, pick up the work where we left off.
Result: If the rubric says "CONTINUE," the model just keeps going. Not a single wasted token. If it says "COMPRESS," you get a summary in three blocks: what's already decided (specific numbers and conclusions are preserved), what's in progress, and what's still open. After the summary the model continues from the current question without drifting back to rejected options.
Why It Works
The LLM weakness: poor memory of its own context. A model doesn't keep an internal log of "what's already settled." It re-digests the entire conversation every single time. When enough old faulty chains pile up, they literally crowd out fresh, correct thinking. The problem isn't length, it's quality: 10 relevant messages beat 40 mixed ones.
The LLM strength: judging against criteria. Ask a model to "tell me when we should pause" and it will fumble. But give it concrete criteria ("the task is finished / we're mid-reasoning") and it applies them precisely. Going from a vague question to a structured rubric is the fundamental shift.
How the method exploits this: The rubric turns the abstract question "is this a good moment to pause?" into a set of checkable conditions. The model doesn't guess; it checks a list. Two conditions for "compress," two for "don't compress" are enough to hit the right moment. A tool without a rubric is chaotic; a rubric without a tool is useless. Together, they work.
Control knobs: - Check frequency — in the study, every N tokens/calls. Done manually: every 8-10 messages or after finishing a meaningful block - Summary format — you can add a "what we dropped and why" block for full transparency - Rubric conditions — extend them for your context (for example, add "is the round of edits finished") - Who triggers it — in the study, the model itself; done manually, you paste the check prompt explicitly
Prompt Template
Evaluate our conversation against the rubric and decide whether we should make a summary now.
COMPRESS if at least one is true:
— We've finished a distinct block of work and have a clear conclusion
— We're already converging on a decision and only details remain
CONTINUE if:
— We're in the middle of a line of reasoning and the thought isn't finished
— We're stuck and repeating ourselves — compression won't help
Answer: COMPRESS or CONTINUE.
If COMPRESS — write a summary:
✅ Decided: {what to lock in — facts, conclusions, numbers}
🔄 In progress: {current open question}
❓ Open questions: {what still needs working through}
After the summary, continue with {current task}.
Placeholders:
- {what to lock in} — specify exactly what to preserve if needed (for example: "key numbers," "decisions made")
- {current open question} — what's being explored right now
- {current task} — which question to resume from after the summary
🚀 Quick start — paste into the chat:
Here is the SELFCOMPACT template for smart context compression.
Adapt it to my task: {your task, for example:
"analyzing a marketplace business model"}.
[paste the template above]
The LLM will ask about the type of task and the logical blocks it breaks into, because the COMPRESS/CONTINUE conditions have to match the structure of your specific work.
Limitations
⚠️ Short conversations: For chats under 15-20 messages you don't need the rubric. The context isn't cluttered yet, and the check just burns tokens.
⚠️ Stuck in a loop: If the model is going in circles, the rubric will say "CONTINUE" (by design: compression won't rescue a stuck model). Loops need a different fix: change the approach or start a new chat.
⚠️ Summary quality: The method is only as good as the model's summarizing. If an important detail gets lost in the summary, there's no getting it back. Check the ✅ block before moving on.
⚠️ Manual mode takes discipline: In the original, the model inserts the rubric itself at intervals. Done manually, you have to remember to paste the check prompt. A habit helps: check every 10 messages or whenever you switch to a new block.
How They Tested It
The Johns Hopkins team asked a simple question: if you give a model a "compress context" tool, will it use it wisely? The answer turned out to be no: some models hit the compress button every two steps, others ignored it entirely. That became the entry point for the rubric.
They tested seven Qwen-family models and three search agents (MiniMax, GLM, MiMo) on six benchmarks: olympiad math (IMO-Answerbench, HMMT) and hard agentic search (BrowseComp). They compared three modes: no compression, a fixed interval (every 30% of context), and SELFCOMPACT with the rubric.
The most telling result is in math on Qwen3.5-9B: no compression 25%, fixed interval 33%, SELFCOMPACT 41%. Nearly a twofold gap between "don't compress" and "compress smart." Here's the interesting part: there was also an "oracle" test, where the model was told when its answer was already correct and compression wasn't needed. The oracle hit 52.9%. That means SELFCOMPACT is still far from the ceiling: the principle is right, and the potential isn't used up.
In search tasks SELFCOMPACT turned out cheaper than the baseline, despite the extra calls. Cost per question dropped by 33-67%, because a compressed context is cheaper to process at every subsequent step.
Adaptations and Extrapolations
🔧 Technique: "A Summary After Every Block of the Task"
If you're working from a structured plan (researching 5 questions in a row), insert the rubric not on a timer but after each closed item. The model will say "COMPRESS" on its own, and you start the next item with a clean, focused context.
We've finished item {N} of our plan.
Before moving on to item {N+1}:
Assess whether we should lock in the outcome in a summary or
the context is still clean enough.
If a summary is needed — write a recap of item {N}
(key facts, decisions, open questions),
then move on to {N+1}.
🔧 Technique: "A Mirror for a Long Project"
An extrapolation to multi-session work (several separate chats). At the end of each session, instead of just closing the chat, run the rubric and save the summary. At the start of the next session, paste it into the system prompt or the first message. SELFCOMPACT becomes a context handoff protocol between sessions.
We're wrapping up the session on {task name}.
Write a recap for the next session:
✅ FINALLY SETTLED: [list with details]
🔄 IN PROGRESS: [what we didn't finish today]
❓ OPEN QUESTIONS: [what needs working through next]
🚫 DON'T DO: [what we already checked and rejected]
The "🚫 DON'T DO" block is the most important:
it keeps us from repeating dead-end paths next time.
Resources
Self-Compacting Language Model Agents (2026, preprint, under review)
Authors: Tianjian Li, Jingyu Zhang, William Jurayj, Xi Wang, Chuanyang Jin (Johns Hopkins University), Mehrdad Farajtabar (Apple), Eric Nalisnick, Daniel Khashabi
Contact: [email protected], [email protected]
Related work mentioned in the paper:
- ReasoningCache (Wu et al., 2026) — a scaffold for compressing reasoning
- Claude /compact command (Anthropic, 2025) — a user-triggered compaction
- Cursor AI compaction (Cursor Research, 2026) — a fixed threshold in the IDE