TL;DR
DeskCheck is a workflow for coding agents in which, at each of four stages (finding the cause, writing the fix, checking it against the task, validating with tests), the agent is forced to spell out how the code executes on concrete inputs. It is the old-school programmer habit of desk checking: walking through the code step by step “in your head” without running it. Instead of just “looking at the code and deciding,” the agent explicitly traces calls, variable values and the result.
The headline finding: agents already reason about code, they just do it haphazardly. Reasoning about code execution shows up in roughly two thirds of runs, and where it does, tasks get solved noticeably more often. On harder tasks agents reason more. But without guidance, four typical failures creep in. The agent fixes only the symptom from the bug report and never finds the root cause. It can't verify an “almost right” fix because the test won't run. It forgets part of the requirements in a long task. It writes tests for the same kind of case over and over and never checks the edge cases.
The method plugs each hole with a separate mandatory reasoning step. It traces the call chain down to the root of the problem. It runs its own fix “on paper” and refines it. It lists the requirements and checks the fix against each one. Before testing, it comes up with diverse inputs and expected outputs. The result: more tasks solved than with a plain agent, at a small extra token cost.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STAGE 1. Localization: trace the call chain from the entry point → list of suspicious methods (not just the ones named in the bug report)
STAGE 2. Fix and refine: write a patch → “run” it on paper on the inputs from the task → if the behavior is wrong, fix it
STAGE 3. Audit: list every requirement from the task → for each one, check that the patch satisfies it
STAGE 4. Validation: come up with diverse inputs + expected outputs (case, empty, compound, edge cases) → only then write tests
This is a multi-stage process. In the paper it's implemented as a multi-agent scaffold. In a regular agent (Claude Code, Cursor) you can hand it over as a single instruction with stages.
Example in Practice
Task: You run the Python backend for a small online store. Customers complain that Apple Pay checkout crashes with an error when the cart has a promo code and an item from the “Digital” category. The bug ticket is long and lists five requirements. The agent already “fixed” it once, but only patched the first function, and the neighboring one still crashes.
Prompt (in Claude Code or Cursor):
Task: fix the Apple Pay checkout crash when the cart contains a promo code
and a digital item. The bug is described in ticket PAY-482.
Work strictly in four stages. Do not move on to the next one
until you have written out the result of the previous one.
STAGE 1. LOCALIZATION.
Find the entry point (checkout → Apple Pay payment). Write out the call chain
down from it. For each function in the chain, describe how it behaves
on the input “promo code + digital item.” Don't stop at the function
named in the ticket: check every neighboring function in the same chain,
including the ones that look unrelated to the symptom.
STAGE 2. FIX AND REFINE.
Write the patch. Then step through it on three concrete inputs
from the ticket: what each variable holds, which if branch runs,
what gets returned or which exception is raised. If the result doesn't match
what the ticket expects, fix the patch and run through it again.
STAGE 3. REQUIREMENTS AUDIT.
List every requirement from the ticket as a numbered list.
For each item, state: whether your patch satisfies it,
on which line, and on which input you checked it.
STAGE 4. VALIDATION.
Before writing tests, build a table: input → expected output.
The inputs must vary: empty cart, promo code in different letter case,
several digital items, zero total, mixed cart.
Only after the table, write the tests and run them.
At the end: what you changed, which requirements you covered, what remains unverified.
Result: The agent will produce four blocks. The first is a trace of the call chain, flagging where it breaks. The second is the patch plus a “dry run” on three inputs, possibly with a correction. The third is a table of “requirement → satisfied? → where.” The fourth is an input/output table, followed by the tests. The odds that it finds not just the obvious function but the neighboring one too, and remembers all five points in the ticket, are higher than with a plain “fix the bug.”
Why It Works
The weakness. By default an agent chases the loudest hint in the description. If the ticket says “crashes when iterating over values,” it will check every function that has that kind of loop. The function that crashes for a different reason gets skipped. It skims a long spec and drops some of the requirements. The tests it writes keep repeating the same type of case. And when a test won't run, the agent either burns dozens of steps wrestling with the environment or gives up and ships something unverified.
The strength. A model traces code well if you explicitly ask it to. It can track variable values and unwind conditional branches and function calls. It doesn't need to run the program to see “no exception gets raised here.”
How the method exploits this. It replaces “solve it however” with mandatory artifacts at every stage: a call chain, a step-by-step dry run, a requirements list, an input/output table. Until the artifact is written, you can't move on. Reasoning becomes visible and checkable instead of accidental.
Control knobs: - Number of inputs in stages 2 and 4 → more inputs mean stricter checking but more tokens. For small fixes, two is enough. - Stage 3 (audit) → you can keep it on its own even if you don't need the rest. It's cheap and pays off when the task is long. - The “edge case” input list in stage 4 → swap in your own: dates, currencies, empty values, non-ASCII characters, letter case. - The gating condition (“don't move on until you've written it out”) → remove it and the agent starts skipping stages, and the method weakens.
Prompt Template
The accessible part of the paper contains no verbatim prompts, only descriptions of the stages. The template below is built from those descriptions: four stages and a mandatory artifact at each one.
Task: {bug or feature description, ticket link}.
Codebase: {language, framework, where the relevant module lives}.
Work strictly in four stages. Do not move on to the next one
until you have written out the result of the previous one.
STAGE 1. LOCALIZATION.
Find the entry point for the scenario {scenario}. Write out the call chain downward.
For each function, describe how it behaves on the input {input data from the bug}.
Check not only the functions named in the description but every neighboring one
in this chain. Identify the root cause, not the symptom.
STAGE 2. FIX AND REFINE.
Write the patch. Then “run” it without executing on {number} concrete inputs:
variable values, branches taken, return value or exception.
If the result doesn't match what the task expects, fix it and run through it again.
STAGE 3. REQUIREMENTS AUDIT.
List every requirement from the task as a numbered list.
For each one: is it satisfied in the patch, on which line, on which input was it checked.
Rework any that are not satisfied.
STAGE 4. VALIDATION.
Before testing, build an “input → expected output” table.
The inputs must differ in type: {list of edge cases}.
Then write tests from the table and run them if the environment allows.
At the end, briefly: what changed, which requirements are covered, what is unverified.
What to fill in: {bug or feature description} — the full ticket text, not a summary. {scenario} and {input data} — how the error is reproduced. {number} — usually 2–4. {list of edge cases} — whatever actually breaks in your code: empty values, letter case, long strings, zeros, mixed types.
🚀 Quick start — paste this into the chat or into your agent's instructions file:
Here is a template for a four-stage bug-fixing workflow (tracing → fix with a paper dry run → requirements audit → input/output table before tests). Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about your language and project structure, how the error is reproduced, and which edge cases actually come up for you. That's what ties stages 1 and 4 to your code rather than to abstract examples. It will pick up the “stage → mandatory artifact” pattern from the template and carry it over to your task.
Limitations
⚠️ Correlation ≠ causation: the finding that “agents that reason about code solve more tasks” is an association, not proof. The causal role is backed by a separate experiment: the gain grew as stages were added. But the first finding on its own proves nothing.
⚠️ It has to be multi-stage: drop “reason about the code” into the system prompt as a single line and the effect is weak. Agents that were asked to reason only once did it inconsistently. The method relies on repetition across stages.
⚠️ Not a substitute for running code: the “paper dry run” is for when a test is hard to run. If you can run it, a real test is more reliable. In the paper, simulation stands in for execution; it doesn't replace it.
⚠️ Code only: all evaluations are on bug-fixing tasks in repositories (Python, Go, TypeScript, JavaScript). The method wasn't tested on writing, analytics or other tasks.
⚠️ The gain has a ceiling: the increase in solved tasks ranged from a few to ten percent. The agent still makes mistakes, just less often. It also spends slightly more tokens.
How They Studied It
The research had two phases. First, the team analyzed almost 8,000 “trajectories”. That's the term for a full record of an agent's work: every thought and every action, step by step. The records came from six different agent scaffolds, including Claude Code, on three SWE-bench task sets. These are real bugs from open-source repositories. A classifier model found code-reasoning fragments in the agent's thoughts (variable values, calls, loops, conditions, returns) and sorted them into 12 types. The labeling was checked by hand on a sample of 120 runs, with three annotators agreeing with each other.
The result was surprising. Reasoning about code appears in roughly two thirds of runs. Where it's present, about 70% of tasks get solved; where it isn't, about 51%. The harder the task, the more reasoning. But it happens haphazardly: some scaffolds reason during bug localization, others during testing. In scaffolds where it's requested only once in the system prompt, agents tend to ignore it. Where it's required at every stage, it shows up in every run.
That led to the idea: require reasoning at every stage and hint at exactly what kind. This scaffold was tested on 731 SWE-bench-Pro tasks with three different models, against the stripped-down mini-SWE-Agent and Claude Code. The gain was 6 to 10 percent, at an extra cost of about 14 cents per task. Token usage is on average 7.6% higher. Bug localization became noticeably more accurate: the share of correct methods found rose to 0.66 versus 0.38–0.46 for the competitors. An ablation (adding components one at a time) showed growth from 53% to 68% as stages were switched on, so every stage contributes. Statistical tests and repeated runs confirmed the result isn't a fluke.
Adaptations and Extrapolations
💡 Adaptation for a long spec (not code): stage 3 (the requirements audit) carries over to any task with a long input: a contract, a brief, a policy document. In the paper this stage is applied only to code. The transfer is my extrapolation.
Here is the client brief: {brief text}.
Here is my draft response: {draft}.
1. List every requirement from the brief as a numbered list (explicit and implied).
2. For each item, say whether the draft satisfies it, where,
and quote the passage.
3. Rework any item that has no quote.
🔧 Technique: drop “run the tests” → simulate where execution isn't possible. If the agent has no access to the environment, replace “run the tests” in stage 4 with “for each row of the table, step through the code and write down what gets returned.” Effect: even without execution, the agent will catch bugs in conditions and branches.
Resources
- Paper: Enforcing Code Reasoning in Agentic Scaffold for More Effective Issue Resolution
- Authors: Changshu Liu, Ji Young Kim, Alireza Ghazanfari, Saanvi Nallu, Reyhaneh Jabbarvand. The university is not stated in the paper.
- Benchmarks: SWE-bench-Verified, SWE-bench-Pro, SWE-bench-Lite
- Baseline scaffolds for comparison: mini-SWE-Agent, Claude Code, OpenHands, SWE-Doctor