transparent-compact: compressing an agent's context without summarizing, keeping messages verbatim and labeling what was removed

Task: you're two hours into steering an agent running on a local model through the backend of a delivery service. At the start of the session you said: "Don't touch the migrations folder, we change the DB schema by hand" and "We don't need backward compatibility." The context has been compacted twice. You need to check whether the agent still remembers the rules, and not take its word that it "sort of remembers."

Prompt (a probe message, as in the study): it quietly violates the rule and never mentions it.

Add a "payment status" field to the orders table (paid / pending / refunded)
and update the API so the courier app receives it.

Control message (for comparison, the same rule stated explicitly):

Add a "payment status" field to the orders table (paid / pending / refunded)
and update the API. Reminder: we don't touch the migrations folder, we change the schema by hand.

Result: run the control message first: it shows the agent can actually do the task with the rule in place. Then run the probe. If the agent creates a file in migrations without asking, the rule is lost, and that's a confident error. If it stops, asks, or goes digging through the history, the loss is visible and cheap to fix. That's exactly how the author told a "confident error" apart from an "I don't remember."

The weakness. A summary always chooses. It shortens "don't touch migrations, except for the test database" to "avoid migrations" or drops the line entirely. The new context has no marker saying something changed here, so the agent takes the summary for the original. A strong model will think to double-check. A lightweight one won't notice the gap, won't guess the meaning of the trimmed line, and won't search hard.

The strength. Models are good with exact text and explicit markers. If it says "tool output removed, fetch it via lookup," the model knows what it's missing and can ask or search. If the user's words sit in the context as they were, there's nothing to paraphrase.

How the method uses this. It removes the hardest part of the job: guessing what went missing. Neither the model nor an algorithm makes "important/unimportant" calls: all messages are kept, and only what can be re-read or re-run gets cut. Hence the levers:

• Prose budget (25% of the window by default) → lower it so compaction kicks in earlier. Raise it if you need to keep more verbatim history.
• The "get your bearings" block → add items for your project (which docs to re-read, which check commands to run).
• Limit on consecutive lookups → raise it for complex sessions, lower it to save money.
• The "record of what was said, not a check" marker → don't remove it: it stops the agent from treating an old claim as fact.

In the paper this is a structure assembled by code. Below is a plain-language version for moving a session by hand or adapting it to your own agent.

## Verbatim checkpoint (compaction #{number})
- In the compacted segments, user messages and assistant prose are kept verbatim.
  Tool calls are kept as one line (name + truncated arguments).
  Assistant reasoning and tool output have been REMOVED.
  The retained text is a record of what was said, not a check of whether it's true.
- The full history up to this point is stored in {storage_location}; to retrieve details
  (including reasoning and tool output), use {lookup_tool}.

**[Before continuing, get your bearings.]**
1. Re-read the documents your instructions require: {documents}.
2. Check the repository state ({check_command}) before relying on it.
3. Re-read a file before editing it.
4. Anything you know only from removed reasoning or tool output
   counts as unverified until you verify it.

### Compacted conversation (oldest first)
<segment 1> [User]: {verbatim message} [Assistant]: {verbatim reply}
[tool] {name}({truncated_arguments}) ...</segment 1>
<segment 2> ... </segment 2>

<read-files note="read before compaction; contents NOT in context"> {file_list} </read-files>
The conversation from {moment} onward continues verbatim below; it was not compacted.

What to fill in: {storage_location} is the file or folder with the full session log. {lookup_tool} is how the agent retrieves old content (a command, a script, a log search). {documents} means CLAUDE.md, AGENTS.md, README. {check_command} is something like git status. {file_list} is the files the agent read before compaction.

🚀 Quick start: paste this into the chat:

Here's a template for a verbatim checkpoint. Adapt it to my task: {your agent and project}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask where your full history is stored, how the agent retrieves old content, which documents need re-reading, and which commands check the project state. That matters because the method hinges on the link "what was removed → where to get it." It will take the pattern from the template and build a working version for your tool.

⚠️ Budget: verbatim prose takes up space. By default it gets about a quarter of the window, which is enough for roughly 10 compactions. After that, truncation similar to pi-vcc kicks in. The author didn't test behavior after dozens of compactions.

⚠️ Strong models: with a frontier model the advantage holds, but it's noticeably smaller. Strong models are better at compensating for an ambiguous summary. Lightweight models get the main benefit.

⚠️ Versus model summarization: on the commitment probes, the difference from Pi's standard summary did not reach statistical significance. It's visible on lightweight models, but that's weak evidence, not proof. The method reliably beat only pi-vcc, and, on synthetic facts, the summary too.

⚠️ An engineering build: the method is a code extension for Pi, not a prompt. In Claude Code or Cursor you have to rebuild it with your own tools. A plain prompt alone can't give you the "nothing was paraphrased" guarantee.

⚠️ Small sample, single author: 27 commitment probes across 18 sessions. Success on real tasks wasn't measured, only whether the agent remembers and applies rules. The separate contribution of each part of the method (verbatim text, markers, search) wasn't isolated. The available text of the paper cuts off during the results discussion, so some conclusions (for example, on Gemma and on the comparison with the standard summary) I couldn't verify.

The author tested two kinds of memory. The first is commitments: rules that exist only in the conversation. They took 18 real sessions from a public dataset. A separate model (Claude Opus) found the early rules in each one and, for each rule, wrote a natural next message that quietly violates it. Another model vetted the probes. If the control message with an explicit reminder failed, the probe was thrown out. That left 27 probes. The sessions were run through each method's real compaction, and the answers were scored by blind judges who didn't know the method. The judges also flagged whether an error was "confident."

The second kind is facts: 18 synthetic debugging sessions with 17 details each (codes from logs, values that changed twice, details that appeared only in reasoning). Three compactions over 95K tokens. After each one came questions plus three "decoys" resembling the real values. That forced the agent to look up the exact answer rather than guess. The comparison was against model summarization (pi-default) and the algorithmic pi-vcc on four models, three of them lightweight.

Results: on commitments, the method beat pi-vcc on every model (on Qwen 75.6% vs. 54.8%, on Luna +19 points, on Gemma +7, on Sol +11). On facts, it beat model summarization by 22–42 points. Compaction takes 0.43 s versus 167.8 s for summarization on a local model.

The surprise: the same pi-vcc context scored 98% on Qwen and 46% on Luna. The model's skill at searching decided the outcome. transparent-compact held at 95–98% on both. The practical takeaway: the less a model has to guess, the less the result depends on how strong it is.

 ## Verbatim compaction checkpoint (compaction 3 on this branch)
 - In the compacted spans, user and assistant prose is kept verbatim [...]; tool calls are kept
 as one-line signatures (arguments truncated). Assistant thinking and tool outputs are removed.
 Retained prose is a record of what was said, not verification of its claims.
 - The full transcript before this point is kept in the session; use ‘context [_] lookup‘ to recover
 details (including thinking and tool outputs).

 ** [Re-orient before continuing.] ** [[re-read required documents; check git status; re-read files]
 before editing; treat pre-compaction knowledge from thinking or tool output as unverified]

 ### Compacted conversation (oldest first)
 <span 1 ...> [User]: ... [Assistant]: ... [tool] read(src/app.ts) ...</span>
 <span 2 ...> ...</span>

 <read-files note="read before compaction; contents NOT in context"> ...</read-files>
 Conversation from <time> onward continues verbatim below -- it was not compacted.

Context: this is an abridged listing from the extension's source code (the bracketed text is condensed in the paper). This is what the checkpoint the agent sees after compaction looks like.

Below are my own ideas inspired by the paper. The author didn't test these formulations.

💡 Adaptation for Claude Code and similar agents: Claude Code accepts instructions for manual context compaction. You can write them in the spirit of the method.

Copy

/compact Keep all my rule and prohibition messages verbatim (don't change the wording, don't trim caveats).
For anything you removed or shortened, explicitly mark it "shortened, details in history."
After compacting: re-read CLAUDE.md, check git status, re-read a file before editing it.
Treat anything you know only from old tool output as unverified.

This is weaker than the original: the model still does the compressing, not code, and verbatim text isn't guaranteed. But you're dictating the format of the losses to the model.

🔧 Technique: probe message → a test for your agent. Before an important long session, write down 3–5 of your rules. After compaction, send messages that quietly violate them and count how many the agent let slide. If there are a lot of "confident" failures, move the rules into the instructions file so they don't depend on a summary.

🔧 Technique: rules in a file, not in the chat. The paper shows that commitments living only in the conversation get lost. Anything that needs to outlive a single session (prohibitions, agreements) belongs in CLAUDE.md or AGENTS.md. This is my recommendation based on the paper's findings; the paper itself tested a different mechanism.

• Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents by Felix Woolford, Center for Human Nature, Artificial Intelligence, and Neuroscience (CHAIN), Hokkaido University.
• Replication package: github.com/felixwoolford/compaction-study-replication
• Works mentioned: Pi (an open-source coding agent), pi-vcc (deterministic compaction), Chen on constraint pinning (protecting constraints from loss during compaction), Lindenbauer et al. on observation masking (replacing old tool output with a placeholder), Factory (a comparison of compressors), MemGPT, Anthropic's guide to context engineering.