TL;DR
AgInt is a way to build an agent so that the harness (plain code wrapped around the model) controls the evidence, not the model itself. The model proposes typed entries for a "certificate" (a dossier): "here's the claim, here's the file and line." The code decides whether to accept the entry, what reliability grade to give it, and whether a verdict can be issued yet. After every step the model gets back the dossier and a list of open obligations.
A regular agent piles up findings in the free-text chat history. What it found, what backs it up, and when to stop are all up to the model. So it declares "checked, all good" without checking, loses findings in a long transcript, and hands down verdicts on hunches. In the paper, a weak model's rate of correctly caught "bad" patches (F1 for the "reject" class) was 0.03 as long as nothing constrained the verdict.
The fix is two rules in code. Step: an entry gets into the dossier only after a deterministic check. Close: an answer is issued only if the dossier discharges the obligations for that answer. The model can't grade itself and can't authorize its own verdict. The dossier survives the session, and you can re-audit it without rerunning the model.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 0: Task → dossier template: slots (questions), grading scale,
requirements "what each answer needs"
(set once, in code)
STEP 1 (loop while budget remains):
Model proposes ONE entry: think / expand / witness
→ Harness: admit or refuse (with a reason)
→ Harness assigns a grade: open → filled → cited → witnessed
→ Model gets back: dossier + list of open obligations
STEP 2: Model proposes a verdict (commit)
→ Close: all obligations discharged? no contradictions?
yes → verdict issued, dossier sealed
no → refusal with a reason, back to STEP 1
Budget runs out with no verdict → "no answer", counted as an error
The steps run inside a single agent loop, but the checks are done by code, not the model.
Example in Practice
The method shines where a verdict has to be backed by facts: accepting a patch, finding the culprit line, confirming that a static-analyzer warning is real. For one-off "explain this code" questions it's overkill.
Task: at an e-commerce marketplace (think an Amazon-scale team), an agent fixes cent rounding on order refunds. You need to decide: accept the patch or not. Typical failure: the agent writes "tests will pass" without looking at which tests are actually affected.
Prompt (system instruction for the agent, an AgInt approximation without harness code):
You are reviewing a patch. Only one answer: ACCEPT or REJECT.
A verdict is allowed only after the dossier is filled in.
DOSSIER (keep it in review_cert.md, append only, never erase):
D1: ACCEPT if all previously failing tests now pass
and all previously passing tests stay green.
P1: what the patch changes
P2: what the tests in the patch change
P3: list of tests that should start passing
P4: list of tests that should stay green
Regions: one per affected test from P3/P4
(at least 2), each with "what it checks" and "why the patch
affects it".
Grade for each slot:
- open: no entries
- filled: your text without a reference
- cited: has a file:line reference
- witnessed: you ran the test and included the output
Rules:
1. One entry per move. Entry first, then the next one.
2. A claim without file:line stays filled.
3. "I ran/reproduced it" without the command output stays cited.
4. If a new fact contradicts an entry with a higher
grade, don't overwrite it; record the conflict.
5. Before the verdict, print a table: slot → grade → what's still open.
6. ACCEPT is forbidden while any region is below cited
or there's an entry where a test fails.
7. REJECT is allowed only if there's a specific region
with a failing test or a test the patch doesn't touch.
Result: the agent fills in the dossier step by step, and before the verdict it prints a "slot → grade → what's still open" table. If it tries to ACCEPT with empty regions, the rules send it back to checking. You end up with a verdict plus a dossier that lets a tech lead see in five minutes what it rests on. Important: in this version the rules are enforced by the agent itself or a human, not by code (see limitations).
Why It Works
The weakness. The model writes the evidence, grades its own evidence, and grants itself permission to rule. Self-checking in free text is weak: "I checked" costs nothing. On top of that, findings sit in a long transcript, and the model loses them instead of using them.
The strength. Models are good at searching: choosing which files to open, forming hypotheses, revising them. Code is good at checking formal things: the reference exists, required slots are filled, the verdict doesn't contradict the findings.
How the method uses this. Search stays with the model, while admission, grading, and verdict permission move to code. A dossier kept as a table of graded slots is persistent memory: after each step the model sees not the transcript but a "what's closed, what isn't" structure. The list of open obligations tells it what to do next.
Control levers:
- Grading scale. Add or remove levels: for example, cited only for references that actually point to the right line.
- Verdict requirements. "Accept" can demand more (all regions witnessed), "reject" just one concrete piece of evidence.
- Number of regions. More required regions means a stricter check and a pricier run.
- Step budget. A limit forces the agent to close the essentials; no verdict by the end of the budget counts as a wrong answer.
- The witness entry. Allow running code, but award witnessed only on real output.
Prompt Template
This is a text approximation of the method for an agent environment (CLAUDE.md, system prompt). In the original, the checks are done by code.
<Role>
You are an analyst-reviewer. Your verdict is accepted only if it is
backed by the dossier. You keep the dossier, but grades and verdict
admission follow the rules below, no exceptions.
</Role>
<Task>
{question_and_allowed_answers}
</Task>
<Certificate>
File: {dossier_path}. Append only; never delete or
rewrite entries.
Slots:
{slot_1}: {what_to_find_out}
{slot_2}: {what_to_find_out}
{slot_3}: {what_to_find_out}
Expansion: you may open a sub-slot under an available slot with
a justification. Maximum {sub_slot_limit}.
</Certificate>
<Grades>
open → filled → cited → witnessed
open: no entries
filled: text without a source reference
cited: has a {reference_format} reference, and it exists
witnessed: has a run result that you showed verbatim
Slot grade = the highest grade among its entries.
A claim "I reproduced it" without output is cited at most.
</Grades>
<Moves>
One move = one entry:
- think(slot, claim, reference)
- expand(slot, sub_slot, justification)
- witness(input, expected) — only if allowed
- commit(verdict)
Reading files is not a move and doesn't change the dossier.
</Moves>
<Requirements>
For verdict "{answer_A}": {which slots at which grade}
For verdict "{answer_B}": {which slots at which grade}
A verdict is forbidden if any entry contradicts it.
</Requirements>
<Loop>
Repeat while budget remains ({number_of_moves} moves):
1. Print the table: slot → grade → what's still open.
2. Pick the weakest slot and make exactly one move.
3. If the move breaks the rules (no slot, no reference,
contradicts a stronger entry), acknowledge the refusal,
state the reason, leave the dossier unchanged.
4. commit only when the table shows every requirement for the
verdict is closed. If not, print what's missing.
Budget runs out without commit → answer "no verdict".
</Loop>
Fill in:
- {question_and_allowed_answers}: for example, "accept the patch: ACCEPT/REJECT";
- slots: the sub-questions without which the verdict can't be trusted;
- {reference_format}: for example, file:line;
- Requirements: what each answer needs.
🚀 Quick start: paste this into a chat:
Here's the AgInt template (a dossier of verifiable evidence). Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which verdicts are possible, which sub-questions each one needs, what counts as a source reference, and whether checks can be run. Those answers shape the slots, the grading scale, and the verdict requirements, and that is the method itself. It will take the pattern from the template and adapt it to your task.
Limitations
⚠️ Only code and execution give guarantees. In the paper, entry admission, grading, and verdict permission are handled by deterministic code. The prompt version above is an approximation: it's checked by the model itself or by you. The model can lie about grades, and the paper's effect hasn't been measured on it.
⚠️ "Cited" doesn't mean "correct." A file:line reference only shows that the line exists. Neither the code nor the model checks that the conclusion actually follows from the cited code. Only
witnessed(a run result) is truly reliable.
⚠️ Code only. Every task in the paper is about code: patch acceptance, culprit-line localization, bug confirmation. The authors didn't test transfer to writing, analytics, or sales.
⚠️ Rigor has a price. Every task needs a well-thought-out template and requirements up front. For simple questions that's overkill.
⚠️ No answer = error. If the budget runs out and the requirements aren't closed, there's no answer. With weak models that happens a lot.
⚠️ Part of the paper is cut off. The full text of the certificate template isn't shown in the available version, so the template above was reconstructed from the description of its structure, not copied.
How They Studied It
The authors took four models (including GLM-5 and Qwen3-Next-80B) and compared three approaches: a plain agent, Agentic Code Reasoning (a previously proposed certificate template without harness rules), and AgInt. They tested on two tasks without code execution. Patch acceptance: SWE-bench-Live and DeepSWE, where the goal is to predict whether a patch passes the tests. Culprit-line localization: Defects4J and GitBug-Java.
Then they added code execution and checked how well AgInt proves bugs: out of 475 null-dereference candidates it confirmed 293, versus 253 for the plain agent. Out of 117 resource-leak candidates, 62 versus 58. Meanwhile, cost per candidate dropped by 59%, and there were 35% fewer executions.
The most interesting part is the matched-template control ablation. The authors took the AgInt scaffold (persistent dossier plus "what's left" hints) and plugged in the old certificate body without the new admission rules. The upshot: in line localization, most of the gain comes from the persistent scaffold itself (the structure, the "what's still open" list). But patch-acceptance calibration is driven by the certificate body and the "gates" in front of the verdict. Practical takeaway: structured memory with a "what's still open" hint buys you a lot even without strict verdict checks.
The authors also verified that the verdict doesn't depend on entry order. They shuffled entries from real runs 25 times, and the decision matched every time. The biggest surprise: the weakest models gain the most. For Qwen3-Next-80B, the "reject"-class F1 rose from 0.032 to 0.496.
Adaptations and Extrapolations
🔧 Technique: drop the budget → spot "stuck" agents
Don't cap the number of moves, and add a line to the loop: "if no grade has changed in 3 moves, explain why and switch hypotheses." You'll see where the agent is spinning its wheels.
🔧 Technique: demand verbatim command output → cut off self-deception
Add: "
witnessedcounts only if you pasted the full command output into a code block." Numbers from "I reproduced it" with no output staycited.
Extrapolation: the same dossier for fact-checking text (not tested by the authors):
You are fact-checking an article. Slots: "claim 1…N".
Grades: open → filled (your opinion) → cited (URL of the primary source)
→ witnessed (a quote from the source in quotation marks).
The "publish" verdict is forbidden while any claim with a number
is below cited. One entry per move; print the table before the verdict.
Resources
- Paper: Agentic Interpretation: Harness-Governed Evidence Construction for Software Agents
- Authors: Yifeng He (University of California, Davis), Yanjun Wang, Siwei Cui, Aritra Sengupta, Murali Krishna Ramanathan (AWS AI Labs)
- Related work: Agentic Code Reasoning (Ugare & Chandra, 2026), SWE-bench-Live, DeepSWE, Defects4J, GitBug-Java