TL;DR
RewardEngine is a setup where the agent writes itself a set of checks before handing in the work. Some checks are hard yes/no tests, others are graded quality rubrics. Each check recomputes the result from the source data and returns a “receipt”: “expected 78%, got 92% — fail”. Then the checks are stress-tested on deliberately bad and borderline examples, frozen, and only after that does the agent revise its work against them.
The core pain: open-ended tasks (a report built from spreadsheets, ticket triage) have no ready-made right answer. A judge model goes by its “overall impression,” so a polished report with a wrong number sails right through. The agent itself doesn't know what to fix either: it hears “better” or “worse,” not “this number doesn't match the source rows.” One more risk: a check the model wrote for itself can be broken. It reads the wrong column or waves through an almost-right answer.
The method boils down to three moves. Build checks from the task and the first drafts. Attack them with counterexamples and fix them without breaking what already worked. Use the frozen set: a critic takes up to three failed checks and proposes edits, and the new version replaces the original only if it is strictly better and fails nothing the original passed.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STAGE 1. BUILD THE REWARD (criteria + checks)
1a. A jury (3 roles) writes commitments from the task, before seeing any results
→ then reconciles them with the first drafts
1b. A reducer merges them into criteria: test (pass/fail) or rubric (fail/partial/full)
1c. A builder attaches a check to each criterion
that recomputes from the source → verdict + receipt
STAGE 1.5. SELF-ATTACK (quality control for the checks)
Build three kinds of examples: control (must pass),
error (violates the criterion), boundary (near miss)
→ a separate "blind" judge assigns the reference label
→ fix a check only if: nothing previously correct breaks,
at least one error gets fixed, no "unresolved" cases remain
→ FREEZE the set
STAGE 2. WORK AGAINST THE REWARD (set is frozen)
Candidate A: task only | Candidate B: edits from failed checks
(critic takes ≤3 checks) | Candidate C: with an extra hypothesis
→ The jury ranks them by results and receipts
→ B or C replaces A only if strictly better AND fails nothing A passed
STAGE 3. UPDATE (only between rounds)
Whatever surfaced during use → goes into the next set of checks
In the original this is a system with many model calls. In a chat or with an agent you can run it as several requests or one long prompt: you get the same structure, no code required.
Example
Task: You sell on Amazon. An agent (Claude Code or Cursor) gets a week of sales export data and has to build a report for the business owner: revenue, keep rate, top 5 SKUs. The classic failure: the report looks gorgeous and shows a 92% keep rate, but the export rows add up to 78%. The file exists, the formatting is on point, and the judge model nods along.
Prompt:
<Role>
You work in "criteria first, work second" mode. You do not hand in the report
until it has passed a frozen set of checks.
</Role>
<Task>
File: amazon_sales_week_42.csv (Amazon sales export, 18,400 rows).
Needed: a weekly report for the owner in report.xlsx — revenue in USD,
keep rate, top 5 SKUs by revenue, 3 takeaways.
Period: October 14–20. Keep rate = units kept / units ordered.
</Task>
<Stage1_BuildReward>
Three roles independently write commitments BEFORE seeing the result:
- Jury_1 (accountant): accuracy of the numbers and the period
- Jury_2 (owner): what is needed to make a decision, what must not be there
- Jury_3 (skeptic): where models usually slip up in reports like this
Then a reducer merges them into criteria. Each criterion:
- type: TESTCASE (pass/fail) or RUBRIC (fail/partial/full with anchor descriptions)
- source: where the commitment came from
- check: how to recompute it from the raw CSV rows (not "looks right")
- verdict format: {criterion, verdict, receipt "expected X, got Y"}
If there is no data to check — verdict "no data" (not fail).
If the check itself breaks — "check not executed".
</Stage1_BuildReward>
<Stage1_5_SelfAttack>
For each check, come up with three kinds of examples:
- control: a correct result (the check must pass)
- error: a result with a violation (for example, a tweaked cell
that changes the keep rate)
- boundary: a near miss (for example, 77.6% instead of 78%, or a period of October 13–20)
Assign each example its reference label separately, without looking at how the check works.
If a check got it wrong — fix it. A fix is accepted only if:
(a) all previously correct cases stay correct, (b) at least one
error case gets fixed, (c) no "no solution" cases appear.
After that the SET IS FROZEN. Do not change the rules until the round ends.
</Stage1_5_SelfAttack>
<Stage2_Execute>
1. Build the report from the task alone → candidate A.
2. Run A through the frozen set, output a table "criterion | verdict | receipt".
3. Critic: pick no more than 3 of the most important failed checks,
write edits that cite the receipts. If unclear — ask a question or abstain.
4. Fix the report → candidate B. Re-check it.
5. Selection: B replaces A only if it is strictly better AND did not fail a single check
that A passed. Otherwise keep A.
</Stage2_Execute>
<Output>
The final report, the check table with receipts,
the list of criteria that remained unchecked.
</Output>
Result: The model first lists criteria from the three “jurors” (for example, the keep-rate formula, the period, reconciling total revenue against the source rows). Then it shows attack examples against its own checks and the fixes to them. Next comes a table of checks run on the first version of the report, with receipts along the lines of “expected / got.” At the end: edits for the three biggest failures, a re-check, a “B replaced A” or “A stays” decision, and a list of what couldn't be verified. The key artifact here is the table with receipts: it shows exactly where the report is lying.
Why It Works
The weakness. When a model grades a result “as a whole,” it judges by appearances: neatness, formatting, a confident tone. A 92% next to a beautiful table looks convincing. If criteria are invented on the fly, they bend to fit the answer that already exists. If a check is never tested, it may be checking the wrong thing.
The strength. Models are good at recomputing something specific from a source (“take these rows, compute the share”) and good at inventing counterexamples. It is easier for a model to find a way to break a check than to write a perfect one on the first try.
How the method exploits this. Criteria are locked in before anyone looks at the result. A check must recompute from the source data and issue a receipt, so the critic knows exactly what to fix. Checks get attacked with counterexamples, and a fix is accepted only with no regressions: whatever was correct stays correct. The set is frozen so that an “improvement” doesn't turn out to be moving the goalposts (swap the ruler and declare things got better).
Prompt control knobs: - Number of checks the critic handles (≤3 in the original) → raise it for big tasks, lower it to save tokens. Fewer means more precise edits. - Jury lineup → give the roles concrete profiles from your field (accountant, lawyer, customer). The criteria get more diverse. - Three kinds of counterexamples (control / error / boundary) → boundary catches “almost right.” Don't drop it: that's the most common real-world error. - The “no data” status → keeps missing evidence from counting as a failure. It separates “violated” from “couldn't verify.” - Replacement condition (“strictly better and broke nothing”) → loosen it if you want more experimentation; tighten it if stability matters.
Prompt Template
<Role>
You work in "criteria first, work second" mode. Do not hand in the result
until it has passed a frozen set of checks.
</Role>
<Task>
{task}
Input data: {files_or_text}
Constraints and definitions: {period, formulas, format, what counts as success}
</Task>
<Stage1_BuildReward>
Three roles independently write commitments BEFORE seeing the result:
- Jury_1 ({role_1}): {what_they_look_at}
- Jury_2 ({role_2}): {what_they_look_at}
- Jury_3 (skeptic): where models usually slip up on a task like this
A reducer merges them into criteria. For each one:
- type: TESTCASE (pass/fail) or RUBRIC (fail/partial/full + a description of each level)
- source of the commitment
- check: how to recompute it from the source data
- verdict: {criterion, verdict, receipt "expected X, got Y"}
No data → "no data". Check broke → "check not executed".
Neither status equals fail.
Keep unresolved commitments visible.
</Stage1_BuildReward>
<Stage1_5_SelfAttack>
For each check, build examples: control (must pass),
error (violates the criterion), boundary (near miss).
Assign reference labels separately, without relying on how the check works.
If a check got it wrong — fix it.
Accept a fix only if:
(1) all previously correct cases stay correct,
(2) at least one error case gets fixed,
(3) there are no "no solution" cases.
After that the SET IS FROZEN.
</Stage1_5_SelfAttack>
<Stage2_Execute>
1. Candidate A: complete the task without leaning on the set yet.
2. Run A through the set → table "criterion | verdict | receipt".
3. Critic: up to {num_checks} of the most important failures; edits that cite
the receipts. If unclear — ask a question or abstain.
4. Candidate B: fix and re-check.
5. B replaces A only if strictly better AND did not fail anything A passed.
</Stage2_Execute>
<Stage3_Update>
If the checking uncovers new defects in the set — log them for the NEXT round.
Do not change the current set.
</Stage3_Update>
<Output>
The result, the check table with receipts, unchecked criteria.
</Output>
What to fill in: {task} — the specific job; {role_1}, {role_2} — two profiles of the people who will sign off on the work; {num_checks} — usually 3.
🚀 Quick start — paste this into the chat:
Here is the RewardEngine template. Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what input data and definitions you have, who signs off on the work, and which mistakes cost you the most. That's how the jury roles and the checks get tied to your real acceptance criteria. From the template it takes the stage structure and fits it to your task.
Limitations
⚠️ Modest effect: on one benchmark the gain in mean score is statistically indistinguishable from zero (the interval includes zero). On the other it is noticeable but small. Don't count on a miracle.
⚠️ Cost: the system makes far more model calls than a single run. The authors deliberately did not compare total cost or equal budgets. Part of the gain may simply be bought with tokens.
⚠️ Doesn't catch forgotten requirements: if no jury role came up with a needed commitment, no check will see it. The authors acknowledge this.
⚠️ Reference labels are model judgments too: their accuracy was not measured. A wrong label gets locked in just as firmly as a right one, because a fix is not allowed to “break” it.
⚠️ Freezing doesn't prevent metric gaming: a fixed check can still be outsmarted by the agent. A stable target is not the same as a correct target.
⚠️ Stability vs quality: the simple “always accept the critic's edit” rule produces more fully successful three-run series than selecting by the set. Selection raises the mean score but gives up a bit of stability.
⚠️ Not tested: the authors did not evaluate reinforcement learning on such rewards, nor a “meta-loop” that improves the way checks themselves are built.
How It Was Studied
The idea was simple: an open-ended task has no right answer, so the agent has to learn the criteria on its own, like a student cramming for an unfamiliar exam. It was tested on two benchmarks: WorkBuddy Bench Office (50 office tasks with mixed files) and Claw-Eval (104 task versions from 68 scenarios with an executable environment). Hidden answers and grader scores were never shown to the system.
The final result (“selected by reward”) was compared against a plain Baseline, against “always take the revision,” against uniform random selection, and against an Oracle that always picks the best candidate by the real grader. On Claw-Eval the mean score rose by ~2 points (the interval excludes zero), the share of “three successes out of three” series grew from 78.4% to 83.7%, and the number of failures dropped by 37.5%. On WorkBuddy, “three out of three” series rose from 45% to 54%, but for the mean score the interval included zero.
They also checked how well the ranking of candidates by the frozen reward matches the grader's ranking. Agreement was about 82% and 80% of pairs. Selecting by reward recovers 26–39% of the gap between random selection and the ideal.
The surprise: the simple “always accept the revision” rule gives the best stability. The practical takeaway: checks with receipts genuinely help an agent fix its output, but choosing between versions remains a trade-off. The authors candidly note that the sets may have “seen” the candidates being evaluated, so generalization to new candidates was not measured.
Adaptations and Extrapolations
🔧 Technique: drop the A/B selection stage → keep only “checklist and fix”
For routine tasks where you only make one version anyway, take Stage1 + Stage1_5 + the “Critic ≤3 → fix” step and remove candidates A/B/C. You get a lightweight version: cheaper, but with less protection against “the fix made it worse.” Bring back the replacement condition if the task is critical.
Extrapolation: a rule in CLAUDE.md / AGENTS.md
Before handing in any report, spreadsheet or document: (1) list the verifiable commitments for the task, splitting hard ones (yes/no) from qualitative ones (on a scale); (2) for each, recompute the value from the source data and write “expected X, got Y”; (3) do not change criteria after seeing the result; (4) fix no more than the three biggest failures; (5) if a fix breaks something that already passed, roll it back; (6) at the end, list what could not be verified.
This carries the paper's principles (criteria before results, receipts, freezing, no regressions) over into a standing instruction for the agent. The paper itself did not test this variant separately.
Resources
- Paper: RewardEngine: Reward-Centric Self-Evolution for Open-Ended Tool Agents. The provided excerpt lists the affiliation of Ji Qi (Department of Statistics and Data Science, CUHK); the full author list is not given.
- Benchmarks: WorkBuddy Bench Office (Tencent Youtu Lab et al., 2026); Claw-Eval (Ye et al., 2026).
- Related work cited in the text: EvoRubrics (Ding et al., 2026) — rubric co-evolution; Self-Refine, Reflexion, CRITIC, TextGrad; Stechly et al., 2025 on how part of the “critique” effect is explained by simply re-asking.