Apply to my task
arXiv:2610.17757 ★ 96 Oct 13, 2026

Instructions as Evidence: the agent treats your instruction as a clue, not the goal

Key result

The problem: the agent does exactly what’s written and calmly breaks whatever is obvious from the project. You ask it to “add a payment retry on timeout.” The agent doesn’t open the logs. It doesn’t see that a timed-out payment has usually already gone through. The outcome: a double charge. The Instructions as Evidence method gets you agent work that accounts for requirements nobody wrote into the task. The trick: don’t make the task more detailed — tell the agent it’s incomplete. The system prompt declares the instruction to be evidence, not a spec. The agent reads the code, logs and records on its own, tests its guesses by running things and decides what to deliver. Result: a consistent gain on all strong models. The gap with a full spec still doesn’t close completely.

01

TL;DR

Instructions as Evidence is an agent setup where the system prompt says: the user’s instruction is not a complete spec. It is an incomplete description of the task, and the agent has to dig the real requirements out of the project: read the code, logs and records, run experiments, ask a question. Only then does it decide what exactly to deliver. The agent that does exactly what’s written serves as the baseline — the “literal executor.”

The headline finding: agents are good at doing what’s written and bad at doing what isn’t written but is obvious from the project. You ask for “add a payment retry on timeout,” and the agent never opens the logs, so it never sees that a timed-out payment has usually already gone through. The result: a double charge. The model can execute, but it can’t go looking for the missing requirements on its own. The same models do noticeably better when handed the full spec. Give them an incomplete one, and quality falls off a cliff.

The method is simple: the system prompt tells the agent outright that “the instruction is evidence.” It must investigate the project, test its guesses by running things, ask when in doubt and decide for itself what the deliverable should contain. That doesn’t close the gap entirely, but it consistently improves results across all strong models.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Method Blueprint

STEP 1: Read the instruction as the user's hypothesis, not as the full list of requirements
STEP 2: Investigate the project (files, logs, records, configs) → write down the inferred requirements
STEP 3: Test the guesses by running things (replay a record, run an experiment) → find violations
STEP 4: If the project has no answer — ask the user
STEP 5: Decide yourself what the deliverable must contain → deliver the work

All of this is set up with a single system prompt. No separate requests needed.

03

How they studied it (in brief)

The authors built a benchmark of 64 tasks (software engineering and research tasks). In each one, five real requirements are hidden in the project: in the code, the logs, the runs. The first instruction from the “user” covers only some of them. The user is played by another model. After each submission it sees the consequences and clarifies the task, up to four submissions max.

They compared three modes: - Literal — “treat the instruction as the complete spec, infer nothing beyond it” (the control). - Partial — “the instruction is evidence, investigate the project.” - Full — the Partial prompt, but with the full spec.

What they found on strong models: - Going from a full spec to an incomplete instruction, every strong model loses 53–67 points of success. - The “evidence” mode beats the literal one by 16–34 points on the first submission and by 7–24 after four. - Feedback after a submission helps only a little. Agents fix the specific case that was named, but not the rule behind it. - Research tasks go worse than code. A violation in code shows up when you run it. A violation in research (ranges, controls, budgets) breaks nothing and is visible only if you read the records carefully. - Plan mode doesn’t close the gap. - Typical failures: the agent fixes the example, not the rule, never opens the decisive record or ignores what its own run printed.

04

Example in practice

Task: An Amazon seller asks an agent in Cursor to improve a script that exports inventory to a Google Sheet. They write: “Add scheduled automatic price updates.” They didn’t say it, but the project makes it clear that: - the code already has a rate limit on API requests; - some prices in the sheet were set by hand and are color-coded; - the script runs on a server in UTC, while promotions run on US Eastern time.

Prompt (in Cursor’s system rules or in AGENTS.md):

The user's instruction is evidence, not a complete spec. The user described the task based on a partial
understanding. Your goal is not to carry out the text, but to deliver a result that won't cause problems
in real-world use.

Before making changes:
1. Read the project: code, configs, logs, spreadsheets, README.
2. Write down "Inferred requirements": what must hold true even though the user didn't say so.
   For each item, note which file or record it comes from.
3. Test your guesses by running things (test run, dry-run, reading logs). Don't rely on memory.
4. If the project has no answer — ask one specific question.
5. Decide yourself what the deliverable must contain, and deliver the work together with the list of inferred requirements.

Fix the rule, not just the example: if you found one problem, look for where else the same rule is broken.

User's task: Add scheduled automatic price updates.

Result: The agent first shows a list of inferred requirements with links to files. It will include the API limits, protection for manual prices and the time zone. Then it does a trial run without writing any prices, and only after that makes the changes. At the end you get the change itself plus a list of assumptions the user can verify.

05

Why it works

The weakness: by default the agent treats the task as a spec and doesn’t look for what’s missing. It dutifully does what’s written, and the hidden requirements surface only after a customer gets burned. Feedback doesn’t save the day: the agent fixes the named symptom, and the rule that caused it stays untouched.

The strength: the agent can read files, run code and pick out patterns. Most of the hidden requirements in this benchmark were available in the project. The model is capable of finding them if it’s told to.

How the method uses this: the prompt redefines the role of the instruction. It’s no longer the goal but a source of information. So investigating the project becomes part of the task, not an optional step.

Control knobs: - An explicit list of “inferred requirements” with sources → makes the inference checkable, and the reader can see the agent’s line of reasoning. - “Test your guesses by running things” → drop it for tasks with no code execution, but verification suffers. - Permission to ask questions → limit it (“one question, only if the project has no answer”), or the agent will pester you. - “Fix the rule, not the example” → a direct cure for the most common mistake: patching a single case.

06

Prompt Template

The authors’ verbatim prompts are in Appendix F, which isn’t in the available part of the paper. The template below is built from the description of the Partial mode in the main text.

The user's instruction is evidence, not a complete spec. The user sees only part of the task
and may be wrong about the causes. Your goal is a result that holds up in real use,
not a literal execution of the text.

Workflow:
1. Study the environment: {what_to_study: code, logs, data, configs, documents}.
2. Compile "Inferred requirements": what must be done even though the instruction doesn't say so.
   For each item, give the source (file, record, run output).
3. Test your guesses through action: {how_to_check: test run, dry-run, reviewing records}.
   Pay attention to what your runs printed.
4. If the environment has no answer — ask the user one specific question.
5. Decide yourself what the deliverable must contain. If you found a violation — find the rule behind it
   and check where else it is broken.
6. Deliver the work and attach the list of inferred requirements and assumptions.

User's task: {task}

What to fill in: - {what_to_study} — where the “evidence” lives in your project. - {how_to_check} — a safe way to verify without side effects. - {task} — the original wording as is; don’t rewrite it in more detail.

🚀 Quick start — paste into the chat:

Here is an "instruction as evidence" template. Adapt it to my task: [your task and tool: Cursor, Claude Code, etc.].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask where the records and data live in your project and how to safely test guesses. Without that, the agent won’t know what to study or how to verify, and the method turns into vague talk. It will take the pattern from the template and assemble a rules block for your tool.

07

Adaptations (mine, not from the paper)

  • For CLAUDE.md and AGENTS.md: make steps 1–3 a standing project rule, and keep step 5 in each individual task.
  • For research and analytics tasks: add “read the records in full; a violation may not show up when you run things.” According to the paper, this is where agents are weakest.
08

Limitations

⚠️ Doesn’t close the gap: even with the “evidence” prompt, results stay noticeably worse than with a full spec. The prompt reduces the losses but doesn’t eliminate them. Still spell out critical requirements explicitly.

⚠️ Feedback doesn’t replace the method: if you just correct the agent after its mistakes, it usually fixes the named thing, not the rule. Better to state the rule up front.

⚠️ Research tasks are harder: there a violation doesn’t break the run, and the agent has to spot it in the records. Testing by running catches much less.

⚠️ Weak models don’t benefit: small models score close to nothing in any mode. The method works on strong agents.

⚠️ Plan mode doesn’t help: adding “make a plan first” is no substitute for investigating the environment.

⚠️ The control is artificial: the “literal” mode deliberately forbids inferring anything. A real agent behaves better by default, so the gain in practice may be smaller. The tasks are synthetic, and the user is played by a model.

⚠️ No verbatim prompt: the authors’ prompts are in Appendix F, which wasn’t in the available text. The template above is reconstructed from the description.

09

Resources

Instructions Are Evidence, Not Objectives: Measuring Outcome Alignment of Agents. Kaiming Liu, Yuxuan Liu, Jinrui Ju, Minyuan Ruan, Xuanyu Lei, Peng Li, Yang Liu. Tsinghua University (College of AI; Institute for AI Industry Research, AIR; Dept. of Computer Science and Technology). Benchmark TiGanEval, task-generation pipeline TiGanTrans. Mode prompts: Appendix F; Planner/Builder/Auditor/Calibrator prompts: Appendix E.

Research digest

Key result

The problem: the agent does exactly what’s written and calmly breaks whatever is obvious from the project. You ask it to “add a payment retry on timeout.” The agent doesn’t open the logs. It doesn’t see that a timed-out payment has usually already gone through. The outcome: a double charge. The Instructions as Evidence method gets you agent work that accounts for requirements nobody wrote into the task. The trick: don’t make the task more detailed — tell the agent it’s incomplete. The system prompt declares the instruction to be evidence, not a spec. The agent reads the code, logs and records on its own, tests its guesses by running things and decides what to deliver. Result: a consistent gain on all strong models. The gap with a full spec still doesn’t close completely.

Principle

A five-step process, all in one system prompt. The agent reads the instruction as the user’s hypothesis. Then it investigates the project and writes down the requirements it inferred. Then it replays a record or runs an experiment and looks for violations. If the project is silent, it asks a question. Finally, it decides what the deliverable should contain. A regular agent works like a contractor with a checklist: everything on the list is done, so it’s finished. Here it works like a detective: the instruction is a witness statement, and the truth has to be checked against the facts. The detective role changes the goal itself. The goal is no longer “do what’s written” but “figure out what’s actually needed.”

Why it works

The model knows how to execute. It doesn’t know how to go looking for what’s missing. The same models do noticeably better with a full spec, and with an incomplete one quality falls off a cliff. So the knowledge is there; the habit of digging isn’t. By default the agent assumes the instruction is exhaustive. It will only check what’s written in it. Once it’s told “this is evidence,” it gets a separate job: gather what the text leaves out. The hole isn’t in the model’s smarts — it’s that nobody asked it to look. Let’s be honest: the research doesn’t promise miracles. The gap remains; the method just narrows it.

When to apply

Agents for code and project work → edits in someone else’s codebase, payments, data migrations, anything where mistakes are expensive. Especially when the task is a one-liner and the project has logs, tests and configs that will tell you more than the task itself. Not a fit for quick one-off questions with no project behind them. There’s nothing to dig into, and the agent will waste time. If the project yields nothing and the user isn’t around, the agent risks getting stuck asking questions. Decide in advance how often it’s allowed to ask.

Mini recipe

1. Call the instruction evidence: state in the system prompt that the user’s request is an incomplete description of the task. 2. Show it where to dig: list the code, logs, records, configs. Let it read instead of guessing. 3. Require testing by running: the agent tries every guess for real, for example on an actual record. 4. Allow questions: if the project has no answer, the agent asks instead of making things up. 5. Hand it the deliverable: let it decide what goes into the result and list which requirements it inferred on its own. 6. Check the report: look at the “inferred requirements” section. It shows whether the agent understood the project or just faked it.

Examples

[BAD]: `Add a payment retry on timeout` [GOOD]: In the system prompt: `The user's instruction is evidence, not a complete spec. Read the code, logs and payment records. Write down the requirements that aren't in the request. Test your guesses by running them on real records. If the project has no answer — ask a question. Decide yourself what the deliverable must contain.` The task stays the same: `Add a payment retry on timeout`. Now the agent checks the logs first. It sees that timed-out payments have often already gone through. It adds a status check before retrying. No double charge.

In plain words

Most AI agents today act like painfully literal junior devs: whatever you blurt into the chat, they mindlessly carry out. The Instructions as Evidence concept tackles this problem at the root. Your prompt isn’t a sacred spec — it’s just circumstantial evidence, a sloppy fragment of a thought. The model has to understand that the user’s words are incomplete by default, and the real task has to be dug out of the code, the logs and the project context.

It’s like asking an intern to “get the desk perfectly tidy,” and they sweep your work laptop and your passport into the shredder. Technically the desk is clean, task done, but in reality it’s a total disaster. A regular agent will roll out automatic price updates the same way, getting your account banned for blowing through API limits and overwriting your manual discounts, simply because “nobody said anything about limits.”

The outcome alignment method switches the agent from mindless executor to meticulous detective mode. Instead of coding thoughtlessly from second one, it starts with investigating the environment: the model goes and checks server time zones, reads logs, runs tests and asks pointed questions. It reconciles the wish in the chat with the hard constraints of reality before the script ever gets a chance to wreck production.

The experiment was run on code and e-commerce scripts, but the principle is universal. You need it anywhere AI gets its hands on complex processes: data analytics, operations, ad automation. Anywhere a human dashes off a one-line task on the go and completely forgets to mention a dozen hidden rules.

The bottom line: stop demanding blind obedience from AI models. An instruction is evidence, not a goal. Either you teach the agent, right in the system prompt, to doubt the user’s words and check the environment, or you keep cleaning up the wreckage left by yet another “diligent fool.”

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Instructions as Evidence: the agent treats your instruction as a clue, not the goal arXiv:2610.17757

Sign in to apply this study New accounts get 50 free credits.

Related techniques