Informed Judge: Check LLM Answers Only Together With the Material They Were Based On

Task: You run a support bot for an online marketplace. It answers customers based on the return policy. A customer asked: "Can I return sneakers I've worn for two weeks?" The bot replied: "Unfortunately, the return conditions cannot be determined from the information provided." Sounds polite. But the policy has a clause about footwear with signs of wear. You need to catch answers like this automatically.

Prompt:

You are checking a support bot's answer for compliance with the policy.

<Policy>
[1] Items of satisfactory quality can be returned within 14 days of receipt.
[2] Footwear is accepted for return only if it shows no signs of wear.
[3] Returned items must retain their original condition and packaging.
[4] Refunds are issued within 10 days after the item is inspected.
</Policy>

<Requirements>
- The answer must rely only on the policy.
- If the policy contains the answer to the question, the bot must give it.
- The bot must not invent conditions that aren't in the policy.
</Requirements>

<Customer question>
Can I return sneakers I've worn for two weeks?
</Customer question>

<Bot answer>
Unfortunately, the return conditions cannot be determined from the information provided.
</Bot answer>

Find all errors in the bot's answer. For each error:
1. Give the number of the policy sentence the answer conflicts with.
2. Quote that sentence.
3. Explain the discrepancy in one sentence.
If you can't provide a quote, don't claim an error.
If there are no errors, write "No errors."

Result: The judge returns a list of claims. Each one has a policy clause number and a quote, for example about the return conditions for footwear. The bot refused to answer even though the policy contained the answer. Response format: a short list, no musings.

Weakness. A judge without the source is "blind" by design. It sees only how the answer reads, not whether it matches the material. A fluent, careful, confident answer looks correct. The judge's intelligence won't help here: you can't recover from a text what isn't in it. That's why a more powerful model without the source loses to a mid-tier one with it.

Strength. LLMs are good at comparing two texts: "does this claim follow from that paragraph?" Give the model both texts and it handles this well. It doesn't need to know the world — it just has to check the answer against the document.

How the method exploits this. The source turns a vague "does it sound good?" into a concrete "is this supported by clause 2?" Numbering and a mandatory quote stop the judge from nitpicking without grounds: there are fewer false positives. The requirements from the instructions remove false claims against answers that actually did everything right.

Control knobs: - Amount of source. For long documents, keep the passages closest to the claims being checked. Junk documents next to the real one cost several points of recall. - Requirements. Listing the requirements explicitly does the most to reduce false positives. If the judge edits answers in a loop, they're mandatory. - Numbering + quote. Turn it on when precision matters: fewer made-up remarks. Turn it off when you need maximum sensitivity. - Reference answer. Helps for tasks with one correct answer (question answering). Hurts for summarization and free-form text — don't add it.

You are checking an answer against the source material and the requirements.

<Source>
[1] {sentence_1}
[2] {sentence_2}
[3] {sentence_3}
...
</Source>

<Requirements>
{list_of_requirements_for_the_answer}
</Requirements>

<Request>
{user_question_or_task}
</Request>

<Answer_to_check>
{answer_being_checked}
</Answer_to_check>

Find every place where the answer contradicts the source, invents something that isn't in it, 
fails to meet the requirements, or refuses to answer when the source contains the answer.

For each problem:
1. The source sentence number or the number of the violated requirement.
2. A quote from it.
3. What the discrepancy is — one sentence.

Rules:
- Rely only on the source, not on your own knowledge.
- No quote — no claim.
- If there are no problems, write "No errors."

What to fill in: {source} — the document, retrieved passages, policy, article. Number the sentences by hand or ask the model to number them. {requirements} — conditions from the system prompt or instructions the answer must meet. {request} — the user's original question. {answer_being_checked} — what you're checking.

🚀 Quick start — paste into the chat:

Here is a template for checking an answer against its source. Adapt it to my task: [your task]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what material the answer being checked was made from and what requirements it had to meet. Without that data the judge is blind again — the template only works with it.

💡 Adaptation for "check and rewrite" loops (self-refine): If you ask a model to critique and rewrite its own text, the critic must see the original task in full, including every constraint. In the study, a critic without the constraints made the result worse. With the authors' annotation and the full set of constraints, the loop came out ahead.

You are a reviewer. Here is the original task with ALL its constraints:
{full_task_with_constraints}

Here is the draft:
{draft}

Check each constraint separately. For each one write: met / violated + a quote from the draft.
Don't suggest edits for constraints that are met.

🔧 Technique: drop the "make it better" → don't lose what's already right. The phrase "improve the answer" without a list of requirements can break what was correct. Only edit the points the judge flagged as violated.

⚠️ A blind judge does damage in an edit loop: if the judge didn't see the constraints and the source, and the system rewrites itself based on its reports, quality falls below the starting point. On instructions with constraints, a loop with such a judge did worse than simply "rewrite it again."

⚠️ Reference answer for summarization: it doesn't replace the source and sharply reduces the benefit of the real source, because a news story has many acceptable retellings. On questions with one correct answer, a reference does help.

⚠️ Extra or false material: a plausible but wrong document sharply increases false positives, and when checking the agents themselves it gave a worse result than no material at all. Don't add things "just in case."

⚠️ Showing the question didn't help: knowing the user's question alone did not improve error detection. What matters is the source and the requirements.

⚠️ A small non-LLM checker model: a small entailment model is good for texts tied to a source. But it fails on paraphrases, and building one yourself takes code.

⚠️ The theory failed quantitatively: most of the authors' numerical predictions didn't hold. The measured effects are reliable; the cybernetic explanation is not.

⚠️ Incomplete text: the available excerpt doesn't show the table of design rules or the details of the closed loop. The conclusions are taken from the introduction and the results table.

The researchers took seven public datasets with human-verified error annotations: RAG assistant answers, news summarization, instruction following, arithmetic, question answering. The focus was on one thing: what exactly the judge is shown. The model, prompt and scoring stayed the same. The same judge received either the answer alone ("blind") or the answer together with its "world" ("informed"). They used three judge models whose prices differed by 10–17x.

The result was surprising: what's in the material matters more than which judge you pick. On RAGTruth, the strongest model without the source got an F1 of 0.217, while a mid-tier one with the source got 0.336. On LLM-AggreFact, the source raised balanced accuracy from 0.57–0.59 to 0.78–0.80.

Next they tested how and how much to show. Ten extra irrelevant documents next to the real one cost 2–5 points of recall. Meanwhile 11% of the source, the part closest to the claims, delivered 98% of the full source's recall. A reference answer next to a news story wiped out about 82% of the precision gain the source provided. Numbering sentences and requiring a quote cut false positives from 3.36 to 2.88 per error-free answer.

The strongest result from the closed loop: a judge without access to the material broke what it was fixing. On instructions, the share of constraints met dropped from 84% to 70%. A simple "rewrite" command without a report got 87%, and a perfect list of violations got 90%. The effect repeated with a second family of judges and when the judge saw the request but not the constraints.

The authors pre-registered 83 testable items and reported honestly: of 81 that were run, 32 cleared the bar, 50 went in the predicted direction, and most of the theory's quantitative predictions didn't hold. So rely on the measured effects.

• Paper: The Anatomy of a Blind Spot: What an LLM Judge Must Be Shown, and How Much
• Authors: Nitish Kumar, Prateik Pothuneedi, Greg Bandak (Hosho, hoshoai.com)
• Theoretical foundation: W. Ross Ashby, An Introduction to Cybernetics; Conant & Ashby, the "good regulator" theorem
• Datasets used: RAGTruth, LLM-AggreFact, IFEval, GSM8K, HaluEval QA