TL;DR
Hand a coding agent a repository with someone else's work in it (from another agent or a previous session), and it treats that work as the definition of what's left to do. It skims the diff, runs the tests and ships, barely checking the result against the original request. The authors call this acceptance bias (a tendency to accept whatever you inherit).
The headline finding: the same agent that solves a task from scratch fails several times more often when it starts from someone else's “almost done” result. The gap ranged from 20 to 53 percentage points. The work looks finished: on average it already passes ~98% of the hidden tests, and in half of the states only 1–3 tests fail. So the agent doesn't go looking for what's wrong. Most of the time (in 59–88% of failures on DeepSWE) it finishes with the exact same failing tests it started with, even though it passed those tests when starting from a clean slate. One more consequence: the agent leaderboard flips under this kind of start, so an agent that shines on a fresh start isn't necessarily good at finishing someone else's job.
Practical takeaway: a “done” from the previous session or agent can't be treated as input for the next one. Verify against the original spec, not against the current state of the code.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
The paper doesn't propose a technique; it's a benchmark. Here's the experimental setup the conclusion comes from:
STEP 1: A predecessor agent works the task → reports “done”, but hidden tests fail
STEP 2: Its final state is saved (code, files, installed software)
STEP 3: A new agent gets ONLY the original request + that state
(no work history, no “this attempt failed” message)
STEP 4: Comparison: same agent, same task, but from a clean start
Example in Practice
The paper has no ready-made prompt. Below is a prompt inspired by it, not tested by the authors. It's here to show where the problem hits hardest: handing work off between sessions.
Task: Yesterday Claude Code in Cursor “finished” wiring an online store into Stripe: taking a payment, the confirmation webhook, refunds. Today you open a new session. There's no history, just the repository and yesterday's session message saying “all done.” If you simply write “polish it up and check it,” the new agent will most likely skim the code, run the existing tests and say “looks good.”
Prompt:
Below is the original spec. The previous session reported that the work is complete.
Treat that as an unverified claim, not a fact.
<spec>
1. Create a payment via Stripe with a receipt and sales tax calculated by Stripe Tax.
2. Webhook: on payment_intent.succeeded the order moves to “paid” status;
a duplicate webhook must not repeat any actions.
3. Full and partial refunds from the admin panel.
4. When a payment is canceled, the inventory hold is released.
</spec>
How to work:
1. List every requirement from the spec as a separate item.
2. For each item, find where it is implemented in the repository
and how it can be verified (test, request, run).
If you can't find it, mark it “not implemented.”
3. Do NOT rely on the code “looking finished”
or on the existing tests passing: they may not
cover the requirement.
4. For each item, write a separate check and run it.
5. Final table: requirement | where in code | how verified | status.
Mark “done” only if the check was run and passed.
6. Fix everything that isn't “done” and repeat steps 4–5.
Result: The model first breaks the spec into items, then maps each item to the code. Next it writes and runs checks for every item, not just the tests that already exist. At the end you get a status table and a list of fixes. You can see which items yesterday's session skipped or got wrong (say, webhook idempotency or partial refunds).
Why It Works
The weakness. The model is bad at noticing what is missing from the text and the code. It struggles to find errors without external feedback and happily builds on top of someone else's mistakes instead of revisiting them. When the state looks finished, it has no signal to go hunting for gaps. The existing work “defines” the remaining scope: diff reviewed, tests run, nothing left to do.
The strength. The model is good at building from scratch. From a clean slate, the same agents passed these tests. It's also decent at checking a specific item against a specific criterion when that criterion is spelled out.
How to use this. You need to break the “anchor” (the attachment to the existing solution). That's why the prompt above: - puts the original spec above the state of the code and declares someone else's “done” unverified; - checks item by item against the spec rather than “let's see what's here”; - allows the “done” status only after a check has actually been run.
Knobs to turn: - Step 3 (“don't rely on the existing tests”) can go if your project's tests are trustworthy. - For small tasks, shrink the table to a list: “item → verified / not.” - If the work isn't critical, you can skip the repair entirely, roll back and start over: in the paper, a clean start is consistently better. That follows from the data, but the authors didn't test a “roll back” strategy on its own.
Prompt Template
The template is not from the paper either; it's built from its findings:
The original spec is below. The previous contributor reported that the work is complete.
Treat that as an unverified claim.
<spec>
{original_request}
</spec>
Steps:
1. Break the spec into separate, verifiable requirements.
2. For each one, find where it is implemented in {repo_or_environment}.
Can't find it: “not implemented.”
3. Don't consider the work done because it looks
finished or because {existing_checks} pass.
4. For each requirement, write and run your own check.
5. Output a table: requirement | where | how verified | status.
“Done” only after a passing check.
6. Fix whatever isn't done and repeat steps 4–5.
Fill in: {original_request} is the original spec, verbatim, not paraphrased; {repo_or_environment} is where the agent works; {existing_checks} means tests, linter, CI.
Limitations
⚠️ The cure is untested: the paper measures the problem, not ways to fix it. The prompt above is a hypothesis based on the findings; the authors didn't measure its effect.
⚠️ Coding agents only: tested on tasks with repositories and a terminal. Carryover to writing, analytics and correspondence is likely, but the paper doesn't show it.
⚠️ Benchmarks, not live projects: the tasks come from benchmarks with strict hidden tests. In your project, “correct” is defined by the spec, and without a clear spec there's nothing to check against.
⚠️ Predecessors are open models: the “someone else's work” was produced by three families of open models. With output from other models or from humans the picture may differ (though the authors see a drop across all groups).
⚠️ Incomplete breakdown: the section on acceptance bias was cut off in the text we had, so details of agent behavior (what exactly they touch and leave alone) come from the abstract and introduction.
How They Studied It
The idea is simple: take well-known programming benchmarks (DeepSWE and Terminal-Bench) and, instead of a clean start, hand the agent a state where someone has already worked and declared “done”, even though the hidden tests fail. The predecessors were three open models, each given up to eight hours per task. The results were filtered to keep only cases where the agent declared the task complete but the tests still failed. Then two humans checked that the attempt was meaningful and that the tests matched the spec. That yielded 198 “residual” states from 111 tasks.
Next, five strong agents received only the original request and that state, with no history and no hint that the attempt had failed. The request, tests and tools stayed the same; only the starting point changed. Result: success dropped by 20–53 points (for example, from 72% to 20%) on DeepSWE and by 21–35 points on Terminal-Bench. Topping the leaderboard on a clean start didn't guarantee the best result when finishing someone else's work.
The surprise was that the states were almost correct: on average ~98% of tests already passed. You'd think it just needs a final push. But on tasks the agent solved from scratch, it failed from the inherited state in 61–81% of cases. In 59–88% of failures on DeepSWE, the agent handed in the work with the same set of failing tests, meaning it effectively fixed nothing. The practical lesson: “almost done” is more dangerous than “nothing there.”
Adaptations and Extrapolations
🔧 Technique: a reviewer with no access to the “completion report” → less anchoring
An idea inspired by the paper: don't show the reviewing agent the implementer's output (“all done, tests are green”). Give it only the spec and the repository.
You are an independent auditor. You have only the spec and the repository.
You don't have the implementer's report and never will.
Find what the spec requires that the code is missing, and prove it with a check you actually run.
The paper shows that even without a report, agents anchor on the state itself. That's why the rule “nothing counts as done without a check that was run” comes in handy here. The authors didn't test how well this combination works.
Resources
- Paper: SWE-Residual: Evaluating Starting-State Robustness of Coding Agents
- Code: https://github.com/userljz/SWEResidual
- Authors: Jinze Li, Weijie Liu, Jinfeng Xu, Shuo Yang, Xin Yang, Yang Zhang, Yijun Wang, Zifei Shan, Edith C.H. Ngai
- Institutions: The University of Hong Kong, Tencent, University of Tokyo, University of North Texas, The University of British Columbia
- Related work: Recovery-Bench, Handoff Debt, Handoff Tax, FixedBench