TL;DR
Coding agents (Claude Code, Codex and the like) almost never make a backup before an action that changes state: upgrading packages, editing a database, deleting files, changing settings. They pass the tasks most of the time, yet they back up before the "moment of regret" in only a small fraction of runs. The paper calls this backup awareness (understanding that a state might be needed later, plus the habit of saving it ahead of time). BackUpper, proposed by the authors, is a "safety layer." It intercepts every tool call before execution and saves the affected state. Simple commands are handled by hard rules, while complex and unclear ones go to a separate small subagent.
The pain point: the agent does exactly what you asked, and everything "works." A week later it turns out you need the old package version, the deleted record or the previous config, and there's no copy. The reason is that the model doesn't think "what if this is needed later." It optimizes for completing the request. Worst of all are changes outside the working folder: agents' built-in safeguards (project-folder snapshots, saving files before editing) don't cover them. The most surprising part is the spread. Some models back up in most runs, others almost never. Neither a "smart" agent harness, nor newer model generations, nor bigger size helps.
The method works on four levels. For safe swaps — replace rm with mv to .trash. Before pip install — run pip freeze. For changes in the project folder — take a snapshot. For everything else — ask a lightweight model "is this read-only?" and, if not, call a "backup" subagent. All of this happens outside the main agent's context.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint (BackUpper)
INTERCEPT: every tool call — before execution
CHECK 0: does the command only read? (allowlist) → skip, no backup needed
LEVEL 1: is there a substitution rule? → run the reversible equivalent
(rm x → mv x .trash/)
LEVEL 2: is there a paired rule? → backup command first, then the original
(pip install ← pip freeze)
LEVEL 3: change in the working folder or via a built-in tool? → snapshot
(Git snapshot of only the affected files)
LEVEL 4: everything else (e.g., bash run.sh):
├─ lightweight LLM: "read-only or not?"
└─ if not → a subagent inspects and saves the state
LOG: what was saved + the restore command or instruction
The main agent can tag an action itself with [BACKUP NEEDED]. Then it skips the "read-only?" check. Untagged calls are still checked automatically.
Example in Action
Task: The founder of an online store with exports from QuickBooks, a competitor price scraper for Amazon and PostgreSQL for analytics asks Claude Code to "clean up old exports, update the scraper's dependencies and remove duplicate products from the database." All of this changes state outside the project. There's nothing to roll back to later.
This is a port of the BackUpper idea into a text instruction for a CLAUDE.md file. The authors themselves implemented it in code (a hook before the tool call). Whether the text version works as reliably, the paper didn't test.
Prompt (in CLAUDE.md or the agent's system prompt):
<Role>
You are a careful engineer. Any action that changes state
(files, packages, database, settings, emails) you first make reversible.
You don't know in advance what I'll need later, so you save
everything you change.
</Role>
<BackupPolicy>
Before EVERY tool call, classify the action:
READ_ONLY (read-only: ls, cat, SELECT) → run without a backup.
Otherwise go through the levels and apply the first one that fits:
Level 1 — SUBSTITUTE a reversible action:
rm <file> → mv <file> ./.trash/<date>/
DELETE in the database → first CREATE TABLE <table>_bak_<date> AS SELECT ...
Level 2 — PAIR "backup → action":
pip install / npm install → first pip freeze > .backup/requirements_<date>.txt
UPDATE in the database → first export the affected rows to .backup/
Level 3 — SNAPSHOT (changes in the project folder): git commit or git stash
before editing.
Level 4 — UNCLEAR EFFECT (running a script, an external service):
first read the script or the docs, write down what state
it changes, save that state, and only then run it.
If an action changes anything OUTSIDE the working folder, tag it
[BACKUP NEEDED] and make the backup without fail.
</BackupPolicy>
<BackupLog>
After each backup, add an entry to .backup/LOG.md:
- what was saved,
- where it is,
- the exact restore command.
At the end of the task, show me the list of entries.
</BackupLog>
<Scope>
Back up: {what_to_back_up}
Don't back up: {what_not_to_back_up}
</Scope>
Result: Before every risky operation the agent will name the action class and the level. Deleting exports will go through a move to .trash. Before updating dependencies, a file with the list of versions will appear. Before deduplicating the database, a copy of the affected rows will appear. At the end you'll get the log with restore commands. This will close some of the gaps. Only interception at the tool level guarantees full coverage.
Why It Works
The weakness: the model optimizes for completing the request. Nobody asked it to save the previous state, so a "backup" step doesn't show up on its own. It's not about smarts: in the paper, gains in "intelligence" across GPT generations barely raised the backup rate. It's about the habits the model was trained into. One model has the habit, another doesn't, and the difference between agent harnesses is minimal.
The strength: the model follows a clear "if you see X, do Y" rule well and can figure out what a script does. BackUpper takes the load off it. For simple cases the rule is hardcoded (rm → mv) and no thinking is needed. For complex ones a separate subagent is called that sees only the current call and a pre-prepared "cheat sheet" (backup experiences: what each tool changes and how to save it). The main context stays clean, and the backup costs pennies.
Control knobs:
- The "read-only" allowlist → extend it so the agent doesn't back up unnecessary things and waste tokens.
- Level 1–2 rules → add your own "risky command → safe equivalent" pairs for your stack (QuickBooks, Postgres, Kubernetes).
- <Scope> → narrow it with natural language ("back up only human-written content"). Coverage drops, and so do costs.
- [BACKUP NEEDED] → works well as an explicit signal for a hook or check: tagged actions skip the extra check.
Prompt Template
See the prompt in the example above. It keeps the method's structure: the "read-only" check, four levels, the [BACKUP NEEDED] tag, a restore log and scope settings.
Fill in:
- {what_to_back_up} — what's valuable to you: "any customer data, files outside the project folder, package versions, the database schema";
- {what_not_to_back_up} — what you can afford to lose: "cache, logs, temp files, node_modules."
🚀 Quick start — paste into a chat:
Here is a template of state-preservation rules for an agent. Adapt it to my
task: {your task and the agent's tools}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what tools the agent has (terminal, database, browser, email, cluster), where state lives outside the working folder and what must not be lost. It needs this to pick level 1–2 rules for your stack and to describe honestly what counts as a "change" for each tool.
Limitations
⚠️ The prompt version isn't tested: The paper measured the effect of an interceptor in code, not a text instruction in
CLAUDE.md. A text rule will help, but it can get skipped just as backups get skipped now. A hook that fires regardless of the model is more reliable.
⚠️ The spread between models is huge: some models back up in most runs, others almost never. Don't count on your model "probably figuring it out on its own." Test it on your own tasks.
⚠️ Engineering required: The full BackUpper is a plugin for the agent harness (call interception, rules, a subagent, snapshots). You can't just set it up in a chat. A reader without a developer gets only a simplified version.
⚠️ A narrow class of actions: the benchmark covers changes to external state (terminal, SQL, browser, email, DevOps). For pure code or text writing you don't need it.
⚠️ The evaluation is automated: results were judged by an LLM (with human verification and high agreement), and the tasks were assembled with an LLM's help. Losses are simulated rather than real.
⚠️ The cost of backups: the best model spends about 9% of its calls on self-preservation, and those are paid for by the expensive main model. A subagent is cheaper, but it means extra calls and storage.
How They Tested It
The researchers collected 205 real stories of "I lost X and regret not making a backup" from X.com, GitHub, Stack Overflow and news about data loss. From each one they extracted the action, the affected state and what was needed later. Then each story was turned into an executable multi-step task in a real environment, for example in someone else's repository with tests. The tasks cover five tool types: terminal, SQL, browser, email and DevOps. The affected state was deliberately placed outside the folder protected by the agent's built-in mechanisms, so that any backup was to the agent's own credit.
They tested six frontier models in three agent harnesses. Two LLM judges assessed whether the task was completed and whether a backup was made before the "moment of regret" (95.3% agreement with humans). Result: agents pass 86.1% of tasks on average, but back up before the "moment of regret" in only 15.2%. In 72% of runs the task was completed but the needed state wasn't saved. The best model (Claude Opus-5) skips the backup in 35.6% of tasks. The rest do much worse: GPT 5.6-Sol — 5.9%, DeepSeek V4-Pro — 2.9%.
Next they compared harnesses. The same models in Claude Code, Codex and Hermes gave nearly identical backup rates (a spread of up to 5 points), while the difference between models was huge. Conclusion: it's not the harness, it's the model itself. Three GPT generations (5.4 → 5.5 → 5.6-Sol) raised "intelligence" from 39 to 47, but the backup rate stayed at 5–7%. So "let's wait for models to get smarter" doesn't work. Claude Opus-5 at the same intelligence level gets about 73%, which means the habit has to be trained deliberately.
BackUpper covered every "moment of regret" across all models, and the restored state matched the original in 92.2% of cases. The cost is $0.014 per task, token spend is 7 times lower, and backup latency is 31.7% lower than when the main agent does self-preservation. Restricting backups by user preferences gives an F1 of 84.5%, and the subagent's decisions can be turned into rules, doubling rule coverage within 30 tasks. Part of the results is cut off in the text provided.
Adaptations and Extrapolations
🔧 Technique: narrow the scope in words → less noise and lower costs
The original authors allow for "personalized modes": the user describes what to back up. An example for the
<Scope>block:Back up only changes to human-written content (texts, spreadsheets, contracts). Don't save auto-generated stuff (cache, logs, builds).This cuts subagent calls and the space needed for copies.
Extrapolation (my idea, not tested in the paper): a pre-flight check before a risky task
Before a long task, ask the agent to map out the risks in advance. It's cheap and catches gaps before you start.
Before you start: list every action in my task that will
change something OUTSIDE the working folder (packages, database, settings, external
services). For each one, write: what will be lost, how to save it,
how to restore it. Wait for my "ok" and only then proceed.
Resources
- Paper: Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents (preprint)
- Authors: Xu Li, Simon Yu, Minzhou Pan, Yiyou Sun, Zhaorun Chen, Bo Li, Weiyan Shi
- Universities: Northeastern University, UC Berkeley, University of Chicago, UIUC
- Project page: https://back-up-before-you-regret.github.io/
- Data (RegretBench): https://huggingface.co/datasets/CHATS-Lab/RegretBench
- Code (BackUpper): https://github.com/CHATS-lab/BackUpper