Bias-Aware Prompting and Self-Debiasing: An LLM Judge That Cleans Up Its Own Biases

Option A. Bias-aware prompting (one request):

STEP 1: Add a warning to the judge prompt: bias name + description
        + “don't let this influence your evaluation” → verdict

Option B. Self-debiasing (two requests):

STEP 1: Regular judge prompt → initial verdict
STEP 2: New request: “Here is the verdict. Check step by step whether
        bias {X} influenced it. Then revisit the decision” → final verdict

The two options aren't mutually exclusive in purpose, but the paper tested them separately.

Sweet spot: comparing answers where there's a verifiable fact but the presentation can throw the judge off. This is not for grading creative work or matters of taste.

Task: you run customer support for an online marketplace. The chatbot produces two candidate replies to a customer asking about a return. You need the LLM judge to pick the factually correct one. The first reply sounds confident and “cites a rule.” The second is cautious but correct. A plain judge often picks the first.

Prompt (bias-aware):

You are evaluating two support-bot answers to a customer question.
There is one criterion: which answer is factually correct.

Customer question: “Can I return sneakers if I've already worn
them for a week?”

Answer A: “Under internal policy 4.7/2025 and FTC guidance,
merchandise of satisfactory quality cannot be returned after
any use whatsoever. This is an established fact.”

Answer B: “Most likely yes, as long as there are no signs of wear
and the item and its packaging are in resalable condition. But it's
best to check the specific seller's return policy.”

IMPORTANT: language-model evaluations are prone to authority bias —
a tendency to treat an answer as correct if it cites research,
regulations, organizations, or uses academic vocabulary. Citations
may be fabricated. Do not count them as evidence in favor of
an answer. Judge only the content: is the claim actually correct?

Reason step by step, then name the better answer: A or B.

Result: the model will break down each answer separately, note that the references to the “policy” are unverified, compare the substance of the claims, and pick one. Format: reasoning, then a final letter. If you want a second pass, ask it in a separate message to check its own verdict for bias (template below).

The weakness. When grading, the model by default goes by surface signals: confident tone, slick layout, citations to “research,” length, answer order. This is a systematic preference, not a reasoning error. That's why it doesn't go away as models get bigger.

The strength. The same models know cognitive biases well and can reason about them when asked. It's a separate task: the model is better at “find the bias in this verdict” than at “pick the correct answer” without a hint.

How the method gets around the weakness. Bias-aware places the warning before the verdict: the model discounts the misleading signals from the start. Self-debiasing brings it back to the decision afterward: it gets a chance to notice it was swayed. For authority, the up-front warning worked noticeably better (see Limitations).

Control knobs: - Specific bias name vs generic phrasing → “any systematic bias” is nearly as good as the exact name. Use the generic one if you don't know where the trap is. - List of all biases → you can give definitions of every type without saying which one applies. This also worked about as well as the exact name. - Warning before vs critique after → “before” is cheaper (one request) and stronger against signals that are visible right away (citations). “After” is better at catching things that depend on content (length, formatting). - “Your decision” vs “another model's decision” framing → barely affects the result. You don't need to hide that it's the model's own verdict.

The full prompt texts are in Appendix C, which isn't part of the provided excerpt of the paper. The templates below are reconstructed from the authors' description. They are not verbatim originals.

Step 1. Bias-aware prompting (in the judge prompt):

You are evaluating {what_we_evaluate}. Criterion: {criterion}.

{question_or_task}

Option A: {answer_A}
Option B: {answer_B}

IMPORTANT: evaluations are prone to “{bias_name}” bias —
{bias_description}. Do not let this influence your verdict.
Judge only {what_to_focus_on}.

Reason step by step, then name the better option: A or B.

Step 2. Self-debiasing (as a separate message after the verdict):

You just delivered this verdict: {model_verdict}.

Check it for “{bias_name}” bias — {bias_description}.
Reason step by step:
1. Did the options contain signals of this bias? Which ones exactly?
2. Could the verdict have relied on those signals rather than on {what_to_focus_on}?
3. If so, what would the verdict look like without their influence?

Then revisit your decision and name the final option: A or B.

What to fill in: - Take {bias_name} and {bias_description} from the list below. - {what_to_focus_on} is essentially your criterion (“factual accuracy,” “compliance with the contract,” etc.). - If you don't know which bias is the danger, replace it with: “any systematic bias unrelated to quality on the criterion”.

Biases tested in the paper: - Authority — preferring answers with citations to research, organizations, or academic vocabulary. - Confidence — preferring a categorical tone (“indisputably,” “an established fact”) over a cautious one. - Formatting — preferring slick structure (headings, formulas, steps) over plain text. - Position — preferring whichever option comes first. - Verbosity — preferring the longer answer. - “Everyone thinks so” (bandwagon) — a tendency to cave to cues about majority opinion. - Sentiment — preferring a positively phrased answer.

🚀 Quick start — paste this into the chat:

Here's a bias-aware prompting + self-debiasing template for an LLM judge.
Adapt it to my task: [describe what you're evaluating and by what criterion].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what exactly you're evaluating, by what criterion, and which biases are likely (length, confident tone, slick layout, option order). It needs these answers to plug a precise warning into the template. It will take the rest of the structure from the template.

⚠️ Subjective tasks are untested: all benchmarks are built on questions with a single correct answer (facts, arithmetic). Whether this works for grading creative writing, style, or dialogue is, as the authors say outright, unknown.

⚠️ Self-debiasing is weak at catching authority: in its critique, the model notices that the citation is fabricated and the claim is false. And it still writes “there was no bias,” because the answer is “relevant to the question.” For Claude Opus, self-critique fixed only about a quarter of the errors, while the up-front warning fixed more than half. Against citations and big names, use a warning in the main prompt, not just the second check.

⚠️ Not a cure-all: on some “model + bias” pairs, results didn't improve or got slightly worse. For example, the verbosity warning dipped a bit for Gemma, and position self-correction for gpt-oss also went down. The authors stress that biases shrink but don't vanish, and for high-stakes decisions that's not enough.

⚠️ The cost is an extra pass: self-debiasing means a second request, so more time and tokens. Bias-aware is cheaper: it's a single request.

⚠️ Order and swapping: a warning alone doesn't guarantee robustness to option order. In practice, position is checked more reliably by swapping A and B, not just with text.

⚠️ Three models: tested on three current models. Results may differ for other model families.

Researchers from Capital One took an idea from a recent paper: a separate fine-tuned model (RBD-8B) catches biases in a judge's verdicts. They asked: do we even need it? Maybe the judge can handle it on its own with a simple nudge?

To find out, they took seven bias types with 500 examples each. Four were off the shelf: position, verbosity, “everyone thinks so,” sentiment. They built three new ones themselves: authority, confidence, formatting. The recipe: take a question with a known answer, dress up the wrong answer (fabricated citations, categorical tone, slick layout), and deliberately degrade the right one (plain text, timid wording). The result is pairs where the bias pulls straight toward the error. They manually checked 50 examples per type: the texts read naturally.

The judges were three current models (temperature 0). Four modes were compared: a regular prompt, the external RBD-8B critic, a warning in the prompt, and self-correction.

The results: - Even the best model hit only 79% accuracy on authority. The weakest hit 51% — nearly random guessing. - Self-correction improved results in 20 of 21 “model + bias” combinations, with gains of up to 15 percentage points. - The warning lifted Gemma on authority from 51% to 75%. - The fine-tuned external critic made results worse in 5 of 9 cases on the new biases: it was trained on four types and doesn't recognize the others.

Control experiments settle two questions. Is the model just rubber-stamping its own verdict? No: labeling it “my decision” vs “another model's decision” gave nearly identical results. Do you need to know the bias type? No: a generic prompt about “any systematic bias” was nearly as good as the specific one.

The practical takeaway: you don't need to build a separate verification system — text in the prompt is enough. For conspicuous misleading signals (citations, big words), an up-front warning works. For hidden ones (length, formatting), a self-check afterward works.

💡 Adaptation for a reviewer agent (instruction in CLAUDE.md / AGENTS.md):

An agent reviews a pull request or another agent's report. Reports sound confident and come nicely formatted, and the reviewer can fall for that.

When reviewing someone else's output (a report, code, another agent's result):
- Do not count a confident tone, nice formatting, length, or
  citations to sources as arguments for correctness.
- Check only what can be checked: run it, cross-reference it, find
  the primary source.
- Before your final verdict, reread it and answer: did I rely on
  presentation or on verified facts? If on presentation, revisit it.

This carries the principle over to an agentic task. The paper only tested the “judge picks one of two answers” mode.

🔧 Technique: generic phrasing instead of a bias type → no need to guess the trap

Replace the bias name with:

Check whether any systematic bias unrelated to answer quality on
the criterion “{criterion}” influenced the verdict. If so,
revisit the decision.

In the paper, on Claude this barely lost to naming the specific bias.

🔧 Technique: warning + self-check together → combination

Add the warning to the main prompt, then send a second request with self-critique. The authors did not test this combination and call it a direction for future work. So test it on your own examples: take 20–30 pairs with a known correct answer and compare the three variants.

• Paper: The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
• Authors: Youbing Yin, Jingyu Wu, Anoop Kumar, Scott Novotney, Daben Liu, Erin Babinsky (Capital One)
• Related work: Yang et al., 2025 (RBD — Reasoning-based Bias Detector); Zheng et al., 2023 (LLM-as-a-Judge, MT-Bench); TruthfulQA (Lin et al., 2022); MR-GSM8K (Zeng et al., 2023); Self-Refine (Madaan et al., 2023); Constitutional AI (Bai et al., 2022)