Information, Not Reruns: What to Give a Stuck Agent So It Finally Fixes the Code

Here’s how the study works. The method is called BRA (Banzhaf Resource Attribution, i.e. breaking down each resource’s contribution):

STEP 1: Take a “frozen” failed state of the agent (task + wrong code)
STEP 2: Clone it into 8 copies — every combination of three resources:
        think (C) / get information (I) / run tests (F)
STEP 3: In every copy — the same single final code edit
STEP 4: Measure the share of hidden tests that pass → contribution of each resource
        on its own and paired with the others
STEP 5: Separately: let the model pick a resource itself and compare with the real benefit

You don’t need to replicate this design. What you need is the practical conclusion:

Agent is stuck
  → NOT “run the tests again” and NOT “think longer”
  → Give it: WHERE to change + WHAT should happen (behavior, cause, edge cases)
  → One edit based on that description

Task: You’re a developer on a Django online store. For the third time, the agent in Claude Code is trying to fix the shipping calculation: free shipping on orders over $50 is checked against the total before the promo code is applied. The test test_free_shipping_after_promo fails. The agent keeps rerunning it and changing something nearby each time.

Prompt:

Stop. Don’t run the tests again and don’t try random edits.
Here is a description of what should happen.

WHERE to change:
- file orders/services/delivery.py, function calculate_delivery_cost()
- called from orders/views.py → CheckoutView.get_context_data()

WHAT should happen:
- Cause of the bug: the free-shipping threshold ($50) is compared
  against cart.subtotal, i.e. the total before the discount.
- Required behavior: the threshold is compared against the total AFTER
  the promo code is applied. A $55 cart with a -$10 promo code = $45 → shipping
  is paid ($5.99).
- Edge cases:
  1) total after the promo code is exactly $50 → shipping is free;
  2) 100% promo code → total is $0, shipping is paid;
  3) no promo code → behavior doesn’t change.

Make one edit based on this description. Then show the diff
and briefly explain how it covers each edge case.

Result: The agent stops trying random edits and makes one targeted edit in the specified function. The reply includes the diff and a walkthrough of the three edge cases. If the spec is right, the test passes on the first try. If it isn’t, you’ll immediately see the gap between your description and what the agent understood.

The weakness. After a failure, the agent doesn’t know what it doesn’t know. Running the tests only gives it the fact that “it failed” plus a chunk of output. It sees a symptom, not a requirement. So it edits code around the symptom and burns turns for nothing. “Think some more” adds no data: the model ponders the same set of facts. Searching the repo brings back a lot of code but doesn’t say what in it should change.

The strength. If the model is told plainly what behavior is required and where to change it, it writes the actual code well. What it lacks isn’t skill but knowledge of the goal. That’s why information keeps its value even for the strongest models, while the value of reasoning drops almost to zero.

How the method exploits this. The spec fills exactly the missing piece: both what and where. The two parts complement each other. A location without understanding the cause gives nothing, the cause without a location gives about half, and both together give the full effect.

Control knobs: - The “where” block (files, functions) → drop it if the location is obvious, but never send it alone. - The “what” block (cause, behavior) → the most important part: it delivers most of the effect. - Edge cases → add them for functions with conditions and thresholds. In the function-level part of the study, the spec included an input–output contract and edge cases. - Length → don’t pad it. Off-topic text of the same length had no effect. Every line should carry a requirement. - “One shot” → ask for one edit based on the description, not an endless loop. That’s how the study did it: exactly one final revision.

Stop. Don’t run the tests again and don’t try random edits.
Below is a description of what should happen.

WHERE to change:
- {files and functions that need to change}

WHAT should happen:
- Cause of the problem: {why it doesn’t work right now — in your own words, no ready-made code}
- Required behavior: {what the code should do — with 1–2 examples using concrete numbers}
- Edge cases: {2–3 cases where it’s easy to get wrong, and the correct result for each}

Make one edit based on this description. Show the diff and briefly explain
how it covers each edge case.

What to fill in: - {files and functions} — where the problem lives. If you don’t know, ask the agent to first show which functions are involved in the scenario. - {cause} — an explanation of “what’s really going on,” not a ready-made patch. In the study, the description didn’t name specific files or variables; it described behavior. - {required behavior} — ideally with real-world numbers: “$55 cart, -$10 promo code → …”. - {edge cases} — thresholds, zero, empty input, “exactly on the boundary.”

For the agent’s instructions file (CLAUDE.md / AGENTS.md) — this is my adaptation, not wording from the paper:

If a test is still failing after two attempts:
1. Don’t run the same test a third time.
2. Spell out exactly what you don’t know about the required behavior
   (where to change it? what’s the correct result? which edge cases?).
3. Ask me those questions and wait for a description.

⚠️ The spec in the study is perfect: the authors wrote it from the reference fix, checking what behavioral information delivers at best. The authors explicitly call the top rungs of the ladder “instruments,” not something the system could obtain on its own. Your spec will be less precise, and a wrong description can lead the agent astray. The paper didn’t measure that risk.

⚠️ Code only: it was tested on Python tasks: functions from MBPP+ and HumanEval+ and fixes in SWE-bench repositories. Carrying it over to writing, analysis or other areas is still a guess.

⚠️ Models without hidden reasoning: all models ran without a built-in “thinking” mode. On models with long internal reasoning, the value of “think more” may turn out differently.

⚠️ Feedback isn’t useless across the board: they only measured a report on public tests, up to five failing cases, and one edit after it. Detailed feedback in real work, where the agent runs the program many times, may behave differently. But the “tests only” choice consistently lost in the study.

⚠️ One edit only: the agent edited once, with no loops. How it all plays out over a long chain of turns, the paper doesn’t test.

⚠️ The paper text was truncated: some results (token economics, a full cost comparison of the resources, the arguments about greedy allocation) aren’t visible in the available excerpt. The conclusions above rely on what’s there.

The idea was simple: instead of comparing whole agents, freeze the moment of failure and see what would actually have helped. The authors took 194 failed functions (first attempts by the small Qwen3-4B model on MBPP+ and HumanEval+) and 126 failed agent fixes on SWE-bench Verified. Each failed state was cloned into 8 copies: every combination of “think / get information / run tests.” Then the same single final edit ran in every copy. Results were checked against hidden tests that never made it into the prompt.

This made it possible to separate the value of the resource itself from side effects like “well, you just made one more edit.” To check whether content matters, they added a placebo: neutral text of the same length. It delivered roughly zero. Information was broken into rungs: repo search, location only, behavior only, and the full spec. That produced a surprising picture: search and “location only” didn’t help, “behavior only” gives about half, and both parts together give the full effect.

Six models (from Qwen3-4B to GPT-5.6 and Claude Sonnet 4.5) showed a common pattern: for weak models, “think” was worth 0.13 in share of tests passed; for the strongest, about 0.01. The value of information drops less (0.06–0.14). On the 24 hardest states, where every model failed, information delivered 0.12–0.25 for each. The decline in the overall sample is explained by a “ceiling”: strong models more often solve tasks without help.

Next, the authors let four strong models choose a resource themselves. Three of the four picked feedback almost every time and got less benefit than random choice would have given. The study pre-registered the hypothesis that the resources reinforce each other, and the data didn’t support it. The resources mostly substitute for each other: 21 of 36 pairwise interactions are significantly negative, and none are significantly positive. In other words, you don’t need several “crutches” at once: the most valuable one is enough.

🔧 Technique: forbid “run it again” → fewer wasted turns

Add this to the start of your message to the agent: “Don’t run the tests until you’ve described in your own words what behavior should result.” This forces the agent to articulate the missing knowledge first. This is my adaptation; the paper has no such trick.

🔧 Technique: strip ready-made names from the description → describe behavior

The authors required the behavior block not to name specific files, functions or variables. It described the cause and the required behavior. Stick to this: that way you give the substance, not a ready-made patch, and the agent doesn’t blindly copy the hint.

Extrapolation: a question instead of picking a resource. If the agent itself asks to “run the tests,” that’s a sign it doesn’t know what you want. Swap its choice for a question (my idea, not measured):

Before proposing the next step, answer three questions:
1) What behavior is expected here? Did I describe it clearly enough?
2) Where in the code does the behavior need to change?
3) Which edge cases are missing from my description?
If you can’t answer any of these — ask me instead of running the tests.

• Paper: Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources (NeurIPS 2026, Long-Context Foundation Models workshop).
• Authors: Saiyue Lyu, Jerrold Huang, Yunbei Zhang, Janet Wang, Zhenting Wang, Yingqiang Ge, Haibo Zhang. Universities: University of British Columbia, University of Toronto, Tulane University, Rutgers University, Kyushu Institute of Technology.
• Data: MBPP+ and HumanEval+ (EvalPlus), SWE-bench Verified.
• Method foundation: the Banzhaf value from cooperative game theory, Möbius decomposition for interactions between resources.