LLM-as-a-judge: how to build a grader you can trust

What LLM-as-a-judge is

LLM-as-a-judge means asking a model to score, rank or compare outputs instead of producing them: grading essays or support replies, picking the best of several drafts, reviewing code or summaries against a standard. Its appeal is scale and a written rationale; its weakness is that the model reads text rather than measuring truth.

A judge prompt has four parts you control, and the studies in this hub test each of them:

• Question format: an absolute score, a pairwise choice, or a pass/fail per criterion.
• Criteria: a rubric with anchored levels, or the model's own idea of quality.
• Output order: reasoning before the verdict, or after it.
• Sampling and isolation: one call or several, all criteria together or one per call.

Why rubrics and comparisons turn an opinion into a measurement

Without criteria, a judge applies its own hidden theory of quality. In the codebook study on user stories, LLM evaluations approached human level only when the prompt carried a codebook: named criteria, a small scale and calibration examples for each level. The same study flags what stays hard: models grade structure and form well and the quality of a rationale poorly.

Criteria also have to discriminate. The Recursive Rubric Decomposition study found that generic LLM-written criteria such as "the answer is helpful" cut GPT-4o's agreement with humans from 55.6% to 42.9%, worse than no rubric, because they fit almost every answer. Splitting each broad criterion until it separates the candidates, dropping duplicates and weighting correlated items raised agreement to 73.3% on JudgeBench. A cheap version of the idea from the RewardBench 2 study: one sentence naming the dimension that matters most added +3 pp of accuracy at no cost.

The question format matters as much as the criteria. In a best-of-N study on Chatbot Arena prompts, a 0-100 score produced only about 20 distinct values, so 67% of comparisons ended in ties and the judge delivered 21% of the value of a perfect choice. Asking "A, B or tie?" cut ties from 60% to 4% and raised that value to 61%.

Design choices that tested well

1. Explain first, score last. In a production judge study, six rounds of rubric rewrites barely moved agreement with experts; swapping two output fields so the reasoning came before the score raised it from 43% to 52%. A score written first becomes a position the explanation has to defend.
2. Hints for what is missing. Judges see what is written and miss omissions. In Analytic Hints, production LLM judges caught only 45-53% of planted bugs in COBOL programs with a generic prompt; feeding the findings of a checklist pass into the prompt as hints raised coverage to as much as 94%, while the models kept only 67% of their own original findings.
3. Several ratings, averaged. In the RewardBench 2 study, eight independent ratings added +9.8 pp and, together with the criterion sentence, +11.9 pp over a 71.7% baseline.
4. One criterion per call. When many criteria are scored in one answer, an early high score pulls the later ones up. The multi-agent evaluation framework cleans the text first, then runs one isolated agent per criterion, each with low, medium and high examples.
5. Your own answer before the candidate's. For checkable tasks, have the judge write its own answer before it reads the one it is grading; the verification-asymmetry study explains why, and the verification prompts page covers the method.

When it helps and when it hurts

Judges work best on checkable criteria with an explicit rubric. Grading written work against a reference answer, GPT-4o reached a 0.98 correlation with instructors in a course-grading study, but only 55% exact agreement, and it graded lower than humans in 39% of cases. For subjective criteria the scale itself matters: a 0-5 scale tracked human ratings better than 0-10 in the scale comparison. The studies disagree on calibration examples: the codebook study treats per-level examples as essential, while the RewardBench 2 study found no reliable gain from them once ratings were averaged.

The counter-evidence is serious. In a code-generation judge study, models approved buggy code that looked right, including code they wrote themselves. In the verification-asymmetry study, three different judge models accepted more than half of deliberately wrong answers: they scored persuasiveness, not correctness. A judge allowed to overrule a majority vote scored 10 points below the majority vote; restricting it to overrule only with a verbatim quote made it the best option tested. A SQL-checking judge wrongly rejected correct answers in 77% of hard cases by inventing rules from unfamiliar data values. Splitting a holistic verdict into separate criteria reduced self-preference bias by 31.5% in a self-preference study, but did not remove it.

The working rule these papers support: use a judge to rank and explain, prefer pairwise or rubric formats, sample more than once, and require evidence (a quote, a recomputation, an independent answer) before a judge's verdict overrides anything that matters. Specific biases are covered on the LLM judge bias page.

Evaluate two versions of {what_we_compare}.

Version A:
{version_A}

Version B:
{version_B}

Selection criterion: {criterion}

First explain how they differ on this criterion.
Then give your verdict: A / B / TIE.
Rate your confidence: 1 (nearly equal) — 5 (clear winner).

Evaluate {object of evaluation} in two stages:

STAGE 1 — Quick check against the checklist:
{checklist of typical problems for your task}

Mark what you found from the checklist.

STAGE 2 — In-depth evaluation with hints:
1. Analyze each problem found from the checklist in detail
2. Find problems the checklist doesn't cover
3. Give specific recommendations

OBJECT OF EVALUATION:
{text/code/idea to evaluate}

Here's the ad post text: {text}.

First, break it down in detail:
- Are there any stretched arguments or exaggerations?
- Are there reputational risks if the post lands differently than intended?
- Is the logic actually convincing, or does the text just sound smooth?

Don't praise nice phrasing — look for substantive problems,
even if the text reads easily.

Don't mention a score until you've finished the breakdown.

Only at the very end, give a final score from 1 to 10.