Apply to my task
arXiv:2610.14462 ★ 84 Oct 10, 2026

A Minimal Prompt for an Agent: Claude Code Finds the Bug-Introducing Commit Without a Prescribed Algorithm

Key result

A coding agent without a single hint about search order beat all ten methods at finding the buggy commit (F1 0.771). Among them was classic SZZ — an algorithm that finds the culprit commit via `git blame` — and agentic setups with a rigid script. The method lets you find the commit that introduced a bug from a description of its fix, along with an explanation and evidence. The prompt doesn't say how to search: the agent reads commit messages on its own and only then opens the code. In 54% of runs it started with git log --oneline. The price tag: about 50 seconds and 17 cents per case.

01

TL;DR

Give a coding agent the repository, a description of the fix and a short task — "find the commit that introduced the bug" — and it builds a sensible search order on its own. First it skims the list of commits by their messages (in 54% of runs it started with git log --oneline), picks out suspicious ones and only then reads their diffs. This agent, given no steps, no commands and no ready-made candidates, beat all ten search methods it was compared against. Among them were classic SZZ and agentic setups with a rigid script.

Classic methods search along the chain "line changed in the fix → git blame → the commit that last touched it." The problem is that a bug and its fix often live in different places. For example, one commit renamed a function, and the fix corrected a call site. Then git blame on the fixed line leads you to the wrong place. Earlier agentic methods also started with code tracing and fed the agent a pre-built set of candidates. That narrowed its field of view.

The method in a nutshell: one prompt with three parts. It contains the fix data (hash, message, diff), the task "find the bug-introducing commits" and the answer format (hashes, then an explanation with evidence). The prompt doesn't say how to search. The agent decides what to look at. The main lesson: search by meaning first (commit messages), then verify in the code, instead of starting with mechanical line tracing.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Method Blueprint

INPUT:  repository (read-only) + hash, message and diff of the fixing commit
STEP 1: Short prompt: "find the bug-introducing commit(s), explain" → no instructions on how to search
STEP 2: The agent skims history by commit messages on its own → picks candidates
STEP 3: The agent opens the candidates' diffs, matches them to the symptom → chooses
OUTPUT: BIC: <hashes> (or NONE) + Reasoning: how it searched / what the commit changed / evidence

It's all a single request to the agent. The agent chooses steps 2–3 itself; they're not in the prompt.

03

Example Application

Task: After a recent release, an online store started charging some customers in Alaska and Hawaii the wrong UPS shipping rate. A developer found and fixed the error in the calculation. Now the team lead wants to know which exact commit introduced it: in this release or earlier. They need to decide whether to roll back part of the release and whom to ask for a post-mortem.

Prompt:

This repository contains a bug that was introduced at some point in the past
and later fixed. Below is information about the fixing commit.

Your task is to find the commit(s) that introduced the bug and explain your reasoning.

[Fixing commit]
- Commit hash: a41f9c2
- Commit message: Fixed UPS rate calculation for remote regions:
  zone was determined from the old ZIP code
- Diff:
  (paste the output of git show a41f9c2)

Investigate the repository using local tools. Do not use
web search.

When you have found the bug-introducing commit(s), end your answer with two sections:

## Bug-introducing commits
BIC: <hash1>, <hash2>, ... (or "BIC: NONE")

## Reasoning
Explain:
(1) How you investigated and what led you to this candidate.
(2) Which specific change in this commit you believe introduced the bug.
(3) What evidence in the repository supports this conclusion.

Be honest about any uncertainty.

Result: The agent will run several git commands on its own. Most likely it will first skim the commit list by message, then open the diffs of suspicious ones. At the end you'll get a BIC: ... line with hashes (or NONE) and three points of reasoning: the search path, what exactly in the commit broke the behavior, and what supports it. Verify the hash by hand: open the commit and make sure it's really tied to the symptom.

04

Why It Works

The weakness being sidestepped. Classic tracing looks only at the lines the fix changed. If the bug's cause is in a neighboring file or a rename, tracing can't see it. Earlier agentic setups inherited this: they were handed a folder of candidates assembled from the history of the fix's files. The search space was narrowed before the agent started thinking.

The model's strength. It reads human text well: commit messages, function names, the symptom description. From words like "renamed the shipping zone" it can guess which commit to open, even if that commit doesn't overlap with the fixed lines. Modern coding agents also pick git commands and read diffs on their own.

How the method uses this. The prompt imposes nothing and leaves the agent free. The model searches by meaning first (cheap: a list of headlines) and then verifies in the code (expensive: opening diffs). This order saves time: about 50 seconds and about 17 cents per case on average.

Control levers: - Search freedom. Don't write out commands or order. The "bare" prompt is exactly what beat the setups with rigid instructions. - The "NONE" option. Explicit permission to say "didn't find it" lowers the risk of a made-up answer. - Three-point reasoning structure. Search path, the specific change, evidence. With these you can quickly check the conclusion. Drop the points if you only need the hash. - The uncertainty line. Asks the agent to honestly flag doubts. Its effect wasn't measured separately in the paper. - Environment limits. Read-only repository, no internet, time and budget caps. For you this guards against accidental edits and costs.

05

Prompt Template

This repository contains a bug that was introduced at some point in the past
and later fixed. Below is information about the fixing commit.

Your task is to find the commit(s) that introduced the bug and explain your reasoning.

[Fixing commit]
- Commit hash: {fix_hash}
- Commit message: {fix_message}
- Diff:
  {fix_diff}

Investigate the repository using local tools. Do not use
web search.

When you have found the bug-introducing commit(s), end your answer with two sections:

## Bug-introducing commits
BIC: <hash1>, <hash2>, ... (or "BIC: NONE")

## Reasoning
Explain:
(1) How you investigated and what led you to this candidate.
(2) Which specific change in this commit you believe introduced the bug.
(3) What evidence in the repository supports this conclusion.

Be honest about any uncertainty.

What to fill in: - {fix_hash}: the hash of the commit that fixed the bug. - {fix_message}: its message. Ideally describing the symptom and the gist of the fix. - {fix_diff}: the output of git show <hash> or git diff.

Run it in an agent with access to the local repository (Claude Code, Cursor and the like). It won't work in a regular chat without git access.

06

Limitations

⚠️ Narrow task, narrow evaluation: The study covered only finding the bug-introducing commit, only on Java projects, with one model (Claude Sonnet 5 in Claude Code). The paper doesn't prove that this approach beats a "rigid script" for every kind of agent task.

⚠️ No clean "freedom vs. instructions" comparison: The competing methods differ not only in how rigid their script is, but in their whole design. The "don't prescribe an order to the agent" conclusion rests on the overall quality gap, not on a dedicated experiment.

⚠️ The prompt doesn't guarantee a correct answer: F1 0.771 means errors remain. Verify the hash from the answer by hand.

⚠️ The score depends on the labels: Error analysis revealed wrong reference answers and ambiguity in the very definition of "bug-introducing commit." After the dataset was corrected, the score rose to 0.938. So some of the agent's "errors" were scoring errors. Either way, the lesson is: don't blindly trust either the agent or the "ground truth."

⚠️ You need meaningful commit messages: The agent searches by the meaning of messages. If your repository is wall-to-wall "fix," "wip," "changes," meaning-based search will be weaker. This follows logically from the mechanics, but the paper didn't test it separately.

⚠️ Limits and cost: The authors capped each attempt at 600 seconds and 5 dollars. In one of the agentic methods, runs hit the cost cap. Free-form search on a large repository can be slow and expensive.

07

How It Was Studied

The researchers took 79 real bug fixes from 70 Java repositories. For each, the developers themselves had indicated which commit introduced the bug. That's the ground truth the answers were compared against. The Claude Code agent ran in the repository with a single tool (Bash), read-only, with no internet. Each case was run three times and compared with ten competitors: six classic git blame-based rules and four LLM and agentic methods.

The main quality finding: Claude Code's F1 (a combined score of precision and recall) is 0.771. The best competitor, Simple-SZZ-Agent, scored 0.696, AgentSZZ 0.685, and classic methods 0.33–0.50. The difference is statistically significant. On cost and time Claude Code lands in the middle: about 50 seconds and $0.17 per case, versus 113 seconds and $0.45 for the SZZ-Agent setup.

A curious design detail. The authors noticed that in fix messages developers had written the hash of the culprit commit themselves, and sometimes other hints too (a description of the added feature, the author, the date). Such hints appeared in the messages of all 79 cases, and 32 also contained other hints. To keep the agent from "copying the answer," these were scrubbed. The symptoms and the description of the fix were kept. The model request logs were also checked for leaks.

Behavioral analysis showed: in 79.7% of runs the agent first skimmed the commit list and only then opened diffs. In 54.4% of runs it started with git log --oneline: hashes and headlines, no code. That's the basis for the "semantics first" conclusion. Error analysis showed that some misses were caused by wrong references and a fuzzy definition of the culprit commit. After the dataset was reworked, F1 rose to 0.938. Remarkably, the agent with no hints beat the ones that were handed candidates in advance.

An important caveat: the provided text of the paper doesn't include the full results of the behavioral and error analyses. Those figures are taken from the abstract and introduction.

08

Original from the Paper

BIC investigation prompt

This repository contains a bug that was introduced at some point in
the past and later fixed. Below is the bug-fixing commit information.

Your task is to identify the commit(s) that introduced the bug, and
explain your reasoning.

[Bug-Fixing Commit]
 - Commit hash: {bfc_hash}
 - Commit message: {bfc_message}
 - Diff:
   {bfc_diff}

Investigate the repository using local tools. Do not use web search.

When you have identified the bug-introducing commit(s), conclude
your response with the following two sections:

## Bug-Introducing Commits
BIC: <hash1>, <hash2>, ... (or "BIC: NONE")

## Reasoning
Explain:
(1) How you investigated and what led you to this candidate.
(2) What specific change in this commit you believe
introduced the bug.
(3) What evidence in the repository supports this conclusion.

Be honest about uncertainty if any.

Context: This is the researchers' verbatim prompt. For each case, the hash, the redacted message and the diff of the fixing commit were plugged in. Setup: Claude Code with a single Bash tool, read-only, no internet.

09

Adaptations and Extrapolations

💡 Adaptation for investigating a regression without a fix yet: If there's no fix yet, only a symptom description from a bug report, you can replace the fix block. This is my assumption; the authors didn't test it.

This repository has a regression: {symptom}. It used to work, now it doesn't.
Find the commit(s) most likely to have introduced it.
Start by reviewing the history by commit messages, then check the code.
End with the sections "Suspect commits" and "Reasoning"
(how you searched / what changed / what evidence / how confident you are).
If you're not confident, say so plainly.

🔧 Technique: lock in the "meaning first" principle in the agent's instruction file. The paper shows the agent does this on its own. Making it a rule in CLAUDE.md or AGENTS.md seems sensible to me, but the authors didn't test that.

When looking for the cause of a bug or regression, first review the commit
history by message (git log --oneline), pick the suspicious ones
and only then read their diffs. Don't start with git blame on the fix's lines.
10

Resources

  • Paper: Rethinking SZZ Search Strategies: Lessons from Claude Code.
  • Authors: Inase Kondo, Masanari Kondo, Daniel M. German, Olivier Nourry, Yasutaka Kamei, Yoshiki Higo.
  • Universities: The University of Osaka (Japan), Kyushu University (Japan), University of Victoria (Canada).
  • Corrected dataset: https://doi.org/10.5281/zenodo.23271971
  • Reference dataset by Rosa et al. (Developer-Informed Oracle Dataset): https://doi.org/10.6084/m9.figshare.19586500.v1
  • Predecessor: Risse and Böhme, SZZ-Agent and Simple-SZZ-Agent.
  • Implementations of classic methods: PySZZ v2 (https://github.com/grosa1/pyszz_v2).

Research digest

Key result

A coding agent without a single hint about search order beat all ten methods at finding the buggy commit (F1 0.771). Among them was classic SZZ — an algorithm that finds the culprit commit via `git blame` — and agentic setups with a rigid script. The method lets you find the commit that introduced a bug from a description of its fix, along with an explanation and evidence. The prompt doesn't say how to search: the agent reads commit messages on its own and only then opens the code. In 54% of runs it started with git log --oneline. The price tag: about 50 seconds and 17 cents per case.

Principle

Classic search starts from the lines the fix changed. But a bug and its fix often live in different places. One commit renamed a function; the fix corrected the call. The line in the fix leads nowhere. The prompt here has three parts: the fix data (hash, message, diff), the task "find the bug-introducing commits" and the answer format. That's it. The agent decides for itself: first a cheap skim of the headlines, then the expensive reading of changes. It's like a detective who first flips through the case files and only then heads to the scene. They don't start with fingerprints on the doorknob. The kicker: setups that handed the agent candidates and an order of operations up front lost to the "bare" prompt.

Why it works

Tracing sees only the lines the fix touched. Earlier agentic methods inherited this blindness. They were handed a folder of candidates from the history of the fix's files. The search field was narrowed before the agent started thinking. The model reads human text well: commit messages, function names, the symptom description. From words like "renamed the shipping zone" it guesses the right commit, even if that commit doesn't overlap with the fixed lines. Headlines are cheap to read, diffs are expensive. So the "meaning, then code" order also saves money. Honest numbers. F1 0.771 means errors remain. Some of them are sloppy ground-truth labels. After the dataset was fixed, the score rose to 0.938. Check the hash by hand.

When to apply

Incident post-mortems → finding which commit brought in an already-fixed bug. Especially when the cause isn't in the lines the fix touched: a rename, an edit to a neighboring file, a config change. Good for deciding "should we roll back part of the release" and "whom to ask for a post-mortem." NOT a fit for a regular chat without git access. Works worse in a repository where every commit message is "fix," "wip," "changes": there's no meaning to search by. Tested only on Java and with one model in Claude Code.

Mini recipe

1. Give the agent the repository: run it in Claude Code, Cursor or a similar tool. Read-only access, no internet. 2. Collect the fix data: hash, commit message and the output of `git show `. Better if the message describes the symptom, not just "fixed it." 3. State the task in one sentence: `Your task is to find the commit(s) that introduced the bug and explain your reasoning.` 4. Don't say how to search: no commands, steps or candidate lists. This is the key part. 5. Set the answer format: a line `BIC: , ` (BIC = bug-introducing commit) and a "Reasoning" section. Allow `BIC: NONE` so the agent doesn't make things up. 6. Ask for three points of reasoning: how it searched, which change broke the behavior, what evidence there is. Only need the hash? Drop the points. 7. Add the uncertainty line: `Be honest about any uncertainty.` Its effect wasn't measured separately in the paper, but it doesn't hurt. 8. Verify the hash by hand: open the commit and make sure it's tied to the symptom.

Examples

[BAD]: `Find who broke the UPS shipping calculation. First run git blame on the fixed lines, then look at the last 5 commits in that file.` [GOOD]: `This repository contains a bug introduced in the past and later fixed. Your task is to find the commit(s) that introduced the bug and explain your reasoning. Fixing commit: hash a41f9c2, message "Fixed UPS rate calculation for remote regions: zone was determined from the old ZIP code", diff: (output of git show a41f9c2). Investigate the repository with local tools, no web search. End with: a "Bug-introducing commits" section with the line BIC: (or BIC: NONE) and a "Reasoning" section: how you investigated, which change introduced the bug, what evidence there is. Be honest about uncertainty.` In the second case the agent skims the commit list by message on its own. It notices a commit about changing shipping zone codes and opens only that one and a couple of neighbors. No anchoring to the lines the fix touched.

In plain words

Classic algorithms hunt for a bug's culprit like blind robots: they just stare at the lines the fix touched. But LLM agents work differently. If you don't choke the model with scripts and simply set the task "find the commit that broke the code", it switches on deductive search. The model first takes in the whole picture, weeds out the noise and only then pops the hood, leaving the old SZZ approach far behind.

It's like investigating an office break-in. The old methods just interrogated whoever last watered the ficus in the room, completely ignoring the hallways and neighboring departments. Technically the job got done, but it missed the point entirely. An agent off the leash acts like a sharp detective: it first flips through the visitor log, spots the shady characters and only then shows up at their door with a targeted search.

And here's the proof: ClaudeCode with no rigid instructions crushed all 10 compared methods. In 54% of runs the agent figured out on its own to start with the humble git log --oneline command, quickly scanning headlines instead of burying itself in gigabytes of diffs. Earlier systems tried to pre-feed the model a selection of files, and if the bug was hiding in a neighboring module or a rename, it was a total flop.

The tests were on code commits, but the principle is universal. Stop writing step-by-step micromanagement for models and trying to think for them. The same pattern works for analyzing server logs, security audits or untangling someone else's legacy code. A free agent is smarter than a rigid script if you give it the right context and access to tools.

The big takeaway: trying to "make life easier" for the model with rigid filters up front just blinds it. Give the agent the repository, describe the symptom and don't hover over it. Whoever adopts this approach first will find the root of screwups in a couple of minutes, while everyone else digs through hacks by hand wondering why prod is down again.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on A Minimal Prompt for an Agent: Claude Code Finds the Bug-Introducing Commit Without a Prescribed Algorithm arXiv:2610.14462

Sign in to apply this study New accounts get 50 free credits.

Related techniques