Worker–Reviewer Loop (HOA): Independent Review of the Result by a Second Agent Session That Can't See the First One's Reasoning

STEP 1: The worker gets the task + acceptance plan + (in later rounds) review issues
       → candidate result (version N)
STEP 2: [if needed] The worker states a specific knowledge gap
       → queries the wiki → takes an entry with its source → revises the result
STEP 3: [if needed] Objective check (tests, compiler, calculation)
       → status + error log
STEP 4: The reviewer (new session, no worker history) checks the "frozen" result
       against the original task and acceptance plan
       → list of issues OR "accepted"
STEP 5: Issues found → back to step 1 (the worker resolves EACH one)
       Accepted AND all objective checks passed → stop
       Budget exhausted → stop

The steps run in separate sessions. A single prompt won't do here: the reviewer's independence is the whole point of the method.

Task: An Amazon seller asks an agent in Cursor to write a unit-economics script. The risk is that the agent will make the numbers look "pretty" on two convenient products, quietly simplify the formula (forgetting returns or sales tax), and report "done."

Prompt (for the reviewer, new session, no chat history with the worker):

<Role>
You are an independent reviewer. You didn't write this code and didn't see how it was made.
Your job is to decide whether the ORIGINAL task was completed in full, not a simplified version of it.
You don't edit the code. You only return issues.
</Role>

<OriginalTask>
A Python script reads an Amazon seller's product export (CSV) and calculates
the unit economics of each item: profit per unit and margin in percent.
Account for: purchase cost, the category referral fee, shipping to the customer,
the return rate (a return means shipping both ways), a 6% sales tax on revenue,
and advertising spend (ACoS). Output: a CSV table with totals and a flag
for products with negative profit.
</OriginalTask>

<AcceptancePlan>
1. Every cost item from the task is included in the formula.
2. Returns are calculated correctly, not subtracted as a fixed amount.
3. The script handles empty values, a zero price, and products with no advertising.
4. There is a test with a hand calculation for at least one product, and it passes.
5. The script solves exactly the task given, not a neighboring one (for example,
   it doesn't calculate gross margin instead of after-tax profit).
</AcceptancePlan>

<Candidate>
The files unit_economics.py and tests/test_unit.py (version 3, read-only).
</Candidate>

<RoundSummary>
The worker reports: "Added ACoS accounting, tests are green."
</RoundSummary>

<Instructions>
- Check the candidate against OriginalTask and AcceptancePlan item by item.
- Run the tests on the unmodified code.
- Look for: missed cases, wrong formulas, testing only convenient data,
  swapping the task for an easier one.
- Return a list of issues: "plan item → what's wrong → how to verify."
- If all items are met, write exactly: ACCEPTED.
</Instructions>

Result: The reviewer returns either a numbered list of issues mapped to the acceptance plan items (for example, where returns are calculated wrong or which case the tests don't cover) or "ACCEPTED." The worker in the main session gets that list, must resolve every item, run the tests, and submit a new version. Review repeats until "ACCEPTED" comes back.

The weakness. A model is bad at checking its own work. Within one session it sees its own reasoning and gets used to it (the "anchoring" effect). If an early step was wrong, it builds everything else on top of it. A simple "check yourself" request without an outside signal often fails. Another risk: the agent swaps the task for a convenient version and considers the goal achieved.

The strength. A model is good at finding mistakes in someone else's finished text, as long as it has the original assignment and clear criteria. It doesn't need to remember how it got there. It just compares "what was asked" with "what came out."

How the method uses this. The reviewer gets only the task, the result, and the criteria. It doesn't see the worker's hidden reasoning, and the result is frozen for it (read-only). So it doesn't repeat someone else's mistakes and doesn't "quietly fix" things. The worker has no right to accept its own work, which closes the "I said it's done" loophole. Control knobs:

• Acceptance plan → the more concrete the items, the sharper the review. Without it, the reviewer checks things "in general."
• Round or budget limit → lower it for simple tasks, raise it for complex ones. The authors showed that for hard problems the benefit grows with the length of the loop.
• Objective gates (tests, calculation, compilation) → add them wherever there's something to run. Review checks meaning; running checks fact.
• What the reviewer sees → don't give it the worker's history or excuses. Only the task, the result, the plan, and a brief round summary.
• The "knowledge gap" role → let the worker look things up only with a specific question, not "find something."

Two prompts in two different sessions.

Session 1 — worker:

<Role>
You are the worker. You build the result and revise it based on review issues.
You cannot accept your own work: only an independent reviewer can do that.
</Role>

<Task>{original_task}</Task>

<AcceptancePlan>
{acceptance_criteria: a numbered list of checkable items}
</AcceptancePlan>

<Tools>{available_tools: code execution, tests, knowledge-base search}</Tools>

<Budget>{limit: number of rounds or time}</Budget>

<PriorFeedback>
{reviewer_issues_and_check_log; in the first round: "none"}
</PriorFeedback>

<Rules>
- Resolve EVERY issue from PriorFeedback and, for each one, state what was changed.
- Don't simplify or replace the task. If you think it needs to be narrowed,
  say so explicitly; don't do it silently.
- If you lack knowledge, formulate a specific question
  {to_knowledge_base_or_source} and cite the source of the answer.
- Run the objective checks ({tests_or_calculation}) before submitting.
- At the end, output the result version and the list of resolved issues.
</Rules>

Session 2 — reviewer (new, no worker history):

<Role>
You are an independent reviewer. You didn't create this result and didn't see
how it was made. You don't edit the result; you only check it.
</Role>

<OriginalTask>{original_task}</OriginalTask>

<AcceptancePlan>{acceptance_criteria}</AcceptancePlan>

<Candidate>{frozen_result — read-only}</Candidate>

<RoundSummary>{brief_round_summary_from_worker}</RoundSummary>

<Instructions>
- Check the result against OriginalTask and AcceptancePlan item by item.
- Run {objective_checks} on the unmodified result.
- Look separately for:
  1. missed cases;
  2. wrong conclusions and calculations;
  3. solving the wrong problem (swapping, simplifying, checking only convenient cases).
- Return a list of issues: "item → what's wrong → how to verify."
- If everything is met and the checks pass, write exactly: ACCEPTED.
</Instructions>

What to fill in: {original_task} — the assignment verbatim, not paraphrased. {acceptance_criteria} — 3–7 checkable items. Each should allow a "met or not" answer. {objective_checks} — anything you can run: tests, recalculating an example, a linter, checking against a spreadsheet. If there's nothing to run, keep only the meaning-level review.

🚀 Quick start — paste into a chat:

Here's a worker–reviewer loop template (worker + independent reviewer).
Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what exactly counts as a completed task and which checks can be run: from that it builds the acceptance plan, and the plan is what determines how sharp the review will be. It will also ask which tools the worker has and how many rounds are allowed. It will carry the template's pattern over to your task.

⚠️ The base model sets the ceiling: the reviewer doesn't add knowledge. For weaker models the gain from the loop was modest: on some olympiads the result barely improved. The strong effect showed up on stronger models.

⚠️ The contribution of each part isn't fully separated: in the experiments, the loop, the wiki, and the verifier worked together. The paper can't say exactly how much the reviewer alone contributes without objective checks.

⚠️ Works best where there's a verifiable result: olympiad problems, proofs, code. The authors didn't test the method on subjective tasks (writing, strategy, creative work).

⚠️ Expensive in time and tokens: the best results came from runs lasting tens and hundreds of hours. For a simple task, such a loop is overkill.

⚠️ The paper's full pipeline is engineering: the Lean formal verifier, a sandbox, a wiki with sources, an automated "runner." Without code you can only reproduce the pattern itself: two sessions, an acceptance plan, running tests.

⚠️ A preprint from the method's authors: the authors themselves call some results (for example, on PutnamBench) preliminary. Some scores were assigned by an LLM judge with human verification.

The team took the most recent 2026 olympiads (math, physics, chemistry, programming) to rule out the problems having leaked into model training. For math they added PutnamBench (672 problems) and Lean-Eval (302 research problems). Math results were accepted only after a proof compiler checked them; the model's word wasn't trusted. Then they compared four levels of scaffolding: a plain API call, a CLI agent (tools + files), agent + wiki + verifier, and the full loop with an independent reviewer. This was repeated on four models: one closed and three open.

On a 50-problem subset of PutnamBench, the closed model went from 3 solved (API) to 46 (agent) and 50 (full loop). The open Kimi K3 rose from 1 to 4 and then to 47. The gap between the open and closed models narrowed from 42 problems to 3. On 100 Lean-Eval problems, the plain agent stalled at 34 after about 30 hours, while the reviewer loop reached 63 in 150 hours: it kept finding solutions where the agent without an outside check kept repeating the same mistakes. The result on weaker models was surprising: DeepSeek V4 Pro and GLM-5.3 improved noticeably less, and on IMO or IPhO in places didn't improve at all. The practical takeaway: independent review turns working time into results, but it doesn't replace the model's knowledge and strength. One interesting choice by the authors: they didn't build a wiki for physics and code, because the model already knows those areas. A wiki is needed where knowledge falls short.

🔧 Technique: add objective gates → the reviewer catches facts, not just opinions

In the paper, for code, the worker and the reviewer write and run tests themselves. Add this to the reviewer prompt:

Before writing any issues, run the code on 3 edge cases
(empty input, zero, a very large value) and include the output.

The reviewer relies on an actual run, not on "looks right."

🔧 Technique: separate meaning-level review from format checks → two sets of issues

Give the reviewer's answer two sections: "Task swapping / logic" and "Technical errors." That way the worker won't lose the most important points among the nitpicks.

Extrapolation: a project wiki in the agent's instructions file

In the paper, the wiki grows from accepted results: after acceptance, a curator extracts reusable pieces from them along with the source and verification status. When working with an agent (Claude Code, Cursor), you can do this as a WIKI.md file next to CLAUDE.md:

After the reviewer writes ACCEPTED, add an entry to WIKI.md:
- what was solved (one line);
- where the result lives (file/function);
- how it was verified (test, calculation);
- date and version.
Before starting a new task, search WIKI.md for similar solutions and reference them
instead of writing from scratch.

This is my extrapolation; the authors tested it only on formal math and chemistry.

• Humanize OlympiadAgents: A Multi-Session Workflow for Olympiad Problem Solving with Solve, Review, Wiki, Verify (preprint)
• Code: https://github.com/humanfia/hoa
• Authors: Zhengyang Zhang, Jing Xiong, Hongzhou Lin, Juihui Chung, Zijian Zhang, Song Han, Sihao Liu, Ligeng Zhu
• Institutions: Tsinghua University, NVIDIA, The University of Hong Kong, MIT, Princeton University, Amazon
• Foundations mentioned: Humanize (Liu et al., 2026), Comparator (Lean FRO), Science Mango, Lean-Eval, PutnamBench