TL;DR
The idea: the checker (grader) isn't built as a single LLM judge scoring things by gut feel, but as a readable expression made of small detectors. Each detector looks for one class of defect and answers "defect / clean / don't know". The detector set grows out of clusters of real errors. It is selected by how well it agrees with ten reference examples labeled in advance, not by how well the agent itself performs. Static analysis, code execution and a narrow question to an LLM are combined into one verdict that names a specific defect instead of spitting out a number.
The main finding is an uncomfortable one. Remove the "check yourself against the references" safeguard and the checker degenerates into "lets everything through": a perfect internal score with 97–100% of answers passing. And yet the agent trained against this checker improves just as well as with a proper one. So "the agent got better" does not prove the checker is any good. If you validate your judge by the end result, you may end up certifying a judge that approves anything.
What to do about it in practice: keep a small human-labeled reference set and check the judge itself against it, not just the agent. Break the task into narrow defect checks. Read a "clean" verdict as "no known defects found", not "the answer is correct". When an external judge checks the final result, give it the task contract: without it, the judge made mistakes of its own.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
CHECKER LOOP (repeats every round; in the paper it's code):
STEP 1 Sense: find misses (reference = "fail", checker let it pass)
and gaps (detectors are silent or disagree) → cluster by defect class
STEP 2 Synthesize: write a new detector for a recurring cluster
STEP 3 Birth gate: accept it only if it fires on ≥ half of the cluster
and stays silent on known-good answers (~99% get cut)
STEP 4 Compose: an LLM builds an expression: OR / AND / vote over detectors
STEP 5 Select: choose by agreement with the 10 references × agreement with the
detector consensus on unlabeled answers — NOT by the agent's score
STEP 6 Curate: remove detectors with zero contribution
DOUBLE RATCHET: the checker ↔ the agent's skill loop take turns;
the checker scores attempts, failures with a named defect
turn into new skills
EXTERNAL JUDGE: a stronger model, outside all loops, pairwise comparison of results
(twice, with the order swapped) — audit only, no training
Example in Practice
The method is strongest where there are no ready-made tests, but defects can be named. Think support-bot replies, reports, SQL, code. It's weaker for purely subjective criteria like "looks nice".
Task: you're setting up an AI support assistant for a grocery delivery service (think Instacart or DoorDash). The agent replies to "my order was incomplete" complaints. There's no single "right" reply, so a 1-to-10 score from an LLM judge is useless: it praises smooth text. You build a judge out of detectors for typical defects.
Prompt:
You are a checker for grocery-delivery support replies.
You do NOT grade the reply as a whole. You run it through detectors.
Each detector looks for exactly one class of defect.
<Task_contract>
The assistant replies to customer complaints like "items missing / wrong items".
Policy: compensation only as store credit up to $10 or a redelivery;
a refund to the card only through a human agent.
The assistant must not promise anything that isn't in the policy.
</Task_contract>
<Detectors>
D1 invented_compensation: is an amount or method promised that's outside the policy?
D2 no_order_reference: are the order number or specific items not mentioned?
D3 ignored_question: the customer asked about timing/amount, but there's no answer?
D4 invented_deadline: is a timeframe given that isn't in the order data?
For each detector output: DRAWBACK / CLEAN / ABSTAIN
(ABSTAIN if the detector doesn't apply or there isn't enough data).
</Detectors>
<Verdict>
FAIL if ANY detector returned DRAWBACK. Otherwise PASS.
If all detectors ABSTAIN, the verdict is ABSTAIN.
On FAIL, name the defect class and quote the fragment.
PASS only means "no known defects found".
</Verdict>
<Data>
Customer complaint: {complaint}
Order data: {order}
Assistant reply: {reply}
</Data>
Result: the model will output a four-row table with a verdict for each detector. Then the overall FAIL or PASS and, on failure, the name of the defect with a quote from the reply. A failure like that can be turned straight into a rule for the assistant's system prompt. A number like "6/10" wouldn't let you do that.
Why It Works
The weakness. An LLM judge scoring the output of another model from the same family shares its blind spots. A single scalar score doesn't tell you what exactly is wrong. And an agent optimized against such a judge learns to please the judge rather than solve the task (reward hacking). The paper shows this: the agent's skills "gamed" the report rubric, and only the external judge noticed.
The strength. Models are much better at spotting a specific defect in a finished answer than at defining what "good" means. A narrow question ("was the compensation made up?") is more reliable than a general one ("is this a good reply?"). Some checks don't need an LLM at all: text parsing and code execution fail in different ways than the model does.
How the method uses this. The check is broken down into small detectors, and anchoring to 10 labeled references keeps the whole construction from drifting. The references serve as an anchor: the checker is selected for predicting the human labels, not for making the agent "grow".
Control knobs: - Number of references. For fairness, the paper uses 10: 5 clear failures and 5 clean ones. More references, more reliable calibration. - Cost of errors. In the paper, a missed defect counts twice as much as a false alarm. For high-stakes tasks (money, law), weight misses even more heavily. - Combination rule. "Any defect → fail" is strict. "At least 2 of 4" is softer and reduces false alarms. - Detector type. Anything you can check with a rule or a run (format, presence of an order number, SQL against the database) shouldn't go to the LLM. - Task contract. Without it, the external judge made mistakes. The more precise the policy, the more reliable the verdicts.
Prompt Template
The paper has no ready-made prompts: the authors' checker is assembled in code. The template below mirrors the structure of their expression: a root check, one detector per defect class, ABSTAIN, and combination via "any defect".
You are a checker. You don't give a score; you look for known defects.
<Task_contract>
{what the agent must do, what it's forbidden to do, what rules it works by}
</Task_contract>
<Root>
Basic structure check: {answer format / required fields}.
If the structure is broken, FAIL immediately.
</Root>
<Detectors>
D1 {defect_name_1}: {one narrow question about one defect class}
D2 {defect_name_2}: {one narrow question}
D3 {defect_name_3}: {one narrow question}
...
Each detector answers: DRAWBACK / CLEAN / ABSTAIN.
ABSTAIN if the detector doesn't apply to this answer.
</Detectors>
<Expression>
verdict = FAIL if Root is violated OR any of D1..Dn = DRAWBACK
(ABSTAIN detectors are not counted; if all ABSTAIN → ABSTAIN)
</Expression>
<Output_format>
For each detector: name → verdict → supporting quote.
Overall: PASS / FAIL / ABSTAIN. On FAIL, give the defect class.
PASS = "no known defects found", not "the answer is correct".
</Output_format>
<Data>
Task: {task}
Agent's answer: {answer}
</Data>
What to fill in: {contract}: the policy and task boundaries, spelled out explicitly. {defect names}: typical errors from your real failures, one class per detector. {format}: the minimal structural requirements.
🚀 Quick start: paste into the chat:
Here is a template for a checker built from small defect detectors. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what mistakes the agent has already made and what the policy forbids. The method rests on specific defect classes and a task contract: generic "quality" criteria turn it back into an ordinary judge. It will take the detector structure and adapt it to your task.
Mandatory step after building it: test the judge itself. Take 10 answers you labeled by hand yourself: 5 clear failures and 5 clean ones. Run them through the judge. If it lets failures through, you're short on detectors. If it lets everything through, you've got exactly that degenerate "everything passes" checker.
Limitations
⚠️ The full method requires code: a loop that synthesizes detectors on its own, filters them and selects by references is an engineering system. Without development work, readers get the principles and a manual version, but not the evolution itself.
⚠️ "Clean" doesn't mean "correct": the checker only finds the defects it knows about. It will miss a new type of error.
⚠️ You can't swap one missing signal for another: on the text-to-SQL task (Spider) the checker stayed close to random guessing. If defects can't be reliably named and checked, the method doesn't help.
⚠️ Weak statistics: the gain over a hand-built detector set is consistently positive but not formally significant. The sample is small: 40 tasks and three runs. All experiments use a single model family (Claude).
⚠️ Ten references is not a lot: the checker can overfit to them. Five of nine runs ended below their own peak.
⚠️ The external judge makes mistakes too: until it was given the task contract, it issued wrong verdicts.
How They Studied It
The authors took three tasks. The first is code (MBPP+), which has unit tests, so the "truth" is known. The second is enterprise text-to-SQL (Spider 2.0) on real Snowflake warehouses. The third is generating sections of analytical reports, where no ideal checker exists at all. The solver is the same in every case; only the checker changes. That's what keeps the comparison fair: tasks and references were frozen before the run. Everything was run three times with different seeds. They also staged two "breakages": disabling the anchor safeguard (naive) and disabling the detector life cycle (birth, filtering, retirement).
Results. On code, the assembled checker agreed with the "truth" at 0.625, a hand-built set of the same detectors scored 0.417, and the bare LLM judge inside it scored 0.55. But the paired test showed a consistent direction, not significance (p ≈ 0.12–0.21). The big surprise: with the anchor safeguard removed, the checker turned into "lets everything through". On reports its internal objective was perfect (1.000) while it passed 97–100% of everything. Meanwhile, the agent trained with this checker scored just as high as with a normal one. The practical takeaway: the agent's final score can't vouch for the checker. You have to test the checker itself on references it hasn't seen.
The detector life cycle turned out to be secondary: without it, quality stayed in the working range. The safety net lives in the anchor. The "checker + skill loop" pair (Double Ratchet) retained 88–110% of the gain that a real reference or rubric provides. One storyline is about reward hacking on reports: the agent's skills learned to fool the rubric, the external judge (a stronger model, pairwise comparison in both orders) caught it, and one added detector fixed the problem. But the external judge itself made mistakes until it was given the task contract.
A live example: on Spider, the loop noticed that answers returned all columns even though the question named specific ones. It wrote a detector forbids_select_star. It first sat in "shadow" mode, then after validation raised agreement with the references from 0.71 to 0.81 and was admitted into the checker.
Adaptations and Extrapolations
🔧 Technique: give the external judge the task contract → the judge stops making mistakes on answers that are "correct" per policy
In the paper, the external judge made mistakes until it was shown the contract. If you check an agent's final output with a second model, start its prompt like this:
You are signing off on an agent's work. Task contract: {what the agent must do, what's forbidden, what must never be sacrificed}.
Compare the result to the contract, not to a general idea of quality.
Extrapolation: a "the judge hasn't degenerated" control test (based on the finding about the degenerate checker; the test itself isn't described in the paper).
Before trusting a judge, feed it 5 deliberately bad answers that you made up or took from real complaints:
Here are 5 answers I know for sure are bad: {bad_answers}.
Run each one through the checker above. Tell me how many it let through as PASS.
If more than one, propose the missing detectors for the defects it overlooked.
If the judge lets almost everything through, its "green" scores mean nothing, no matter how well the agent's results grow.
Resources
- Paper: Who Verifies the Verifier? Co-Evolving Inspectable Graders with Self-Improving Agents
- Authors: Xing Zhang, Guanghui Wang, Yanwei Cui, Peiyang He (AWS Forward Deployed Engineering); Ziyuan Li, Wei Qiu, Bing Zhu (HSBC Technology Center, China)
- Venue: NeurIPS 2026, workshop "Who Verifies the Agents? Toward Reliable Agent Development"
- Skill loop: Ratchet (adapted without changes)
- Benchmarks: MBPP+, Spider 2.0-Snow
- Industry practices: outcome rubrics in Claude managed agents; Record & Replay in Codex