TL;DR
Every time you ask an LLM to edit a document, it risks damaging it — not all at once and not obviously, but suddenly and silently. The study found that even the best models (GPT 5, Claude Opus, Gemini Pro) lose or distort an average of 25% of a document's content by the 20th editing round. Weaker models lose more than 50%.
The real trap isn't gradual wear and tear. 80% of all damage comes from rare catastrophic failures: the model works flawlessly for 7 rounds, then drops 30 items in a single request. You don't notice, because the text still reads fine — part of the content has just quietly vanished or changed. This is called silent corruption.
And it compounds: the longer the document, the more irrelevant files in context, the longer the session runs — the faster the damage piles up. A model's early performance does not predict what happens 10 rounds later.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
How the Problem Unfolds
Rounds 1–3: Model performs great → everything looks OK
Rounds 4–7: Small, invisible errors start piling up
Round 8: CATASTROPHIC FAILURE → 10–30% of content lost
(reads like normal text, but some data is gone)
Rounds 9–12: Looks normal again → you assume it's all fixed
Round 13: Another sudden failure → damage accumulates
...
Round 20: 75% of the original survives (best models)
Safest territory: code and configs (Python is the only domain where most models stay stable).
Disaster zone: natural language, niche formats, structured records.
Example in Practice
Task: You're putting together a 15-page investment memo for a startup. You ask Claude to refine it step by step: first rewrite the market section, then add the financial model, then rework the risks, then polish the executive summary... All in one long chat.
What actually happens:
By the 10th request in that chat, the model has most likely distorted or deleted some data from the earlier sections. Numbers may have shifted. Arguments in the risk section may have partially "dissolved." References to competitors may have disappeared. Everything looks polished, but it's no longer the same document.
Checker prompt (paste it in after every 3–4 editing rounds):
Compare the current version of the document with the original below.
Find everything that:
— was in the original and is missing from the current version
— changed in meaning (not style, but substance: numbers, facts, logic)
— was added but contradicts the original
Response format:
1. DELETED: [what exactly and where]
2. CHANGED IN MEANING: [what and how]
3. CONFLICT WITH ORIGINAL: [what was added and why it contradicts]
Original document:
[paste the original]
Current version:
[paste the current version]
Result: The model returns a structured list of differences between the original and the current version. Some will be your intentional edits — others will turn out to be silent decay.
Why This Works (and Why the Problem Is Real)
An LLM doesn't "edit" a document — it regenerates it. When you ask it to change one paragraph, the model usually produces a fresh version of the whole document, using the previous one as a template. Every "regeneration" is a chance for something from the original not to make it into the new version.
Context accumulates and creates pressure. The longer the chat, the more information sits in the model's context. Irrelevant files, earlier versions of the document, the edit history — all of it adds "noise". The model starts losing track of which version is current and what needs to be preserved.
Catastrophic failures aren't a bug, they're a pattern. The study showed that models don't degrade evenly. They hold up, then suddenly "break" in a single request. That's because small accumulated contradictions in context eventually get resolved abruptly — the model "picks" one version of reality and loses the other.
Levers for managing the risk: - Session length → the fewer rounds in one chat, the safer - Context size → remove irrelevant files — every 1,000 tokens of extra context speeds up decay - Task type → structured output (tables, code, JSON) is more reliable than prose
Prompt Template: Document Checkpoint
Use it every 3–4 editing rounds or before an important save:
You are an audit editor. Your job is to find degradation, not improvements.
ORIGINAL DOCUMENT (reference):
{original_text}
CURRENT VERSION:
{current_text}
Check against three criteria:
1. LOSSES: What is in the original but missing from the current version?
(Facts, numbers, arguments, examples — not style)
2. DISTORTIONS: What changed in meaning without my instruction?
(A number is different, a claim got flipped, a conclusion changed)
3. STATUS: Document intact / losses found / critical distortions
Do not comment on writing quality — only on whether the content is preserved.
Placeholders:
- {original_text} — the original before any editing (keep it in a separate chat or a note)
- {current_text} — what you ended up with after a series of edits
🚀 Quick start — paste into the chat:
Here is a template for checking that a document survived LLM editing intact.
Adapt it to my task: {your task — what you're editing}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what counts as the reference (which version is the original) and what kind of content is critical (numbers, facts, structure) — because that determines where the check should focus.
Limitations
⚠️ Diagnosis, not protection: The checker finds decay after the fact. It doesn't prevent losses — it only helps you catch them.
⚠️ The checker itself isn't perfect: LLM-as-a-judge is bad at catching subtle semantic changes. The study found that GPT 5.4 as a judge flags at most 25% of real discrepancies. Mission-critical documents still need a human review.
⚠️ Code is the exception: LLMs edit Python and other programming formats far more reliably. The rules above apply mainly to text documents.
⚠️ Agent modes don't help: If you use AI agents with tools (Operator, auto-editing features), they perform worse, not better, because of context growth and chains of tool calls.
Practical Rules for Working with Documents
These aren't prompts, they're changes in behavior that lower the risk of decay:
🔴 Dangerous: - Editing one document over 10+ rounds in the same chat - Keeping irrelevant files in context "just in case" - Trusting the result without checking just because it "looks fine"
🟢 Safe: - New editing task = new chat (clean context) - Keep the original document outside the chat — in notes or a separate file - Check for preservation after every 3–4 edits - Ask the model to make changes surgically, at a specified location, instead of rewriting everything
How They Studied It
The idea was elegant: if you ask a model to make a change and then undo it, a perfect model returns the exact original. The researchers built 310 work environments across 52 professional domains — from accounting and music notation to crystallography and subtitles. For each domain they created pairs of instructions: "split the file by category" + "merge it back into one file," "convert the amounts to euros" + "convert them back to the original currency." That let them measure quality automatically without manual labeling — just compare the final document to the original.
Then they ran 19 models through chains of 20 such paired rounds and measured how much of the document "survived." The result was worse than expected: even GPT 5.4 and Claude 4.6 Opus lost an average of 25% of the content by round 20 — and the average across all 19 models was 50%. The pattern of critical failures was the surprise: the researchers expected "death by a thousand cuts" and got "rare but severe wounds" instead — 80% of all damage came from isolated catastrophic failures.
The counterintuitive part: giving models tools (search-and-replace, code execution) made things worse, not better. Tools inflate the context 2–5x, and long context is the main enemy of accuracy. In the end, the only domain where most models worked reliably was Python. Everything else is a risk zone.
Adaptations and Extrapolations
💡 Adaptation: The "Session and Snapshot" Rule
Instead of one long chat — short sessions with checkpoints:
Before we start, save this to a separate document (or the top of the chat):
SNAPSHOT v1.0 from [date]:
[paste the full current version of the document]
---
Now make the following change: {specific edit}
After answering, save the new version as SNAPSHOT v1.1.
It's a manual versioning system right inside the chat. Not high-tech, but a working way to keep a restore point.
🔧 Technique: Surgical Edits Instead of Full Regeneration
Instead of "rewrite the section" — "replace this specific spot":
❌ "Rewrite the risks section — make it more specific"
✅ "In the 'Risks' section, second paragraph — replace only the first sentence.
Original: [paste the sentence]
Replace with: [new version]
Leave the rest of the text untouched."
The more precise your instruction about what not to change, the lower the chance of accidentally losing content elsewhere.
🔧 Technique: An Explicit Ban on "Improvements"
Change only what I specified.
Don't improve the style. Don't fix other parts.
Don't add anything of your own to sections I didn't mention.
Return the full document with a single change: {what exactly}.
By default, LLMs love to "improve" things — and that's part of what drives the decay.
Resources
LLMs Corrupt Your Documents When You Delegate — Philippe Laban, Tobias Schnabel, Jennifer Neville, Microsoft Research. Preprint, under review.
Code and dataset: github.com/microsoft/DELEGATE52 · huggingface.co/datasets/microsoft/DELEGATE52