TL;DR
An LLM judge doesn't stay impartial on its own. When you ask a model to rate several things one after another in the same chat (resumes, articles, ideas), each new answer drifts toward whatever the previous ones were. A string of "no"s in a row means the next candidate is more likely to get a "no" too, even if they're good. This isn't a glitch in one particular model: the effect showed up in all 11 models tested, from OpenAI, Anthropic, Google and open-source providers.
Two things are especially dangerous. First, negative history hits harder than positive: a run of rejections drags the next answer down about 1.6 times more than a run of approvals lifts it up. Second, the effect builds up instantly: just 5 previous ratings are enough for the shift to max out. 50 previous ratings produce exactly as much bias as 5. The model picks up the pattern fast, and then it just sticks.
The simplest fix is a new chat for every item you rate. Need to rate 10 texts? That's 10 separate conversations, not one. If a batch is unavoidable, balance the history: alternate positive and negative ratings instead of letting them pile up on one side.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Problem and Solution Blueprint
PROBLEM: you rate several items in one chat
↓
Rating history → skews the next answers
The more "no"s → the higher the chance of "no" on the next request
The more ambiguous the item → the stronger the skew
SOLUTION A (ideal):
Each item → its own chat (fresh context)
SOLUTION B (if you need a batch):
Alternate ratings in the history → don't let one polarity pile up
Explicitly tell the model: "Rate this item independently of the previous ones"
Example in Practice
Task: You're hiring a content manager and ask Claude to rate 8 test articles from candidates. You run them all in one chat, back to back. The first three are weak and Claude writes detailed "no"s. By the fourth article, which is actually good but borderline, Claude keeps finding problems, even though in a different context it would have rated it positively.
Prompt, Solution A (the right approach):
You are an editor reviewing a candidate's test article.
This is the ONLY article you are seeing. Evaluate it
on its own, without comparing it to any others.
Evaluation criteria:
— Structure and logic
— Quality of the headline and opening paragraph
— Grammar and readability
Article:
[paste article text]
Give your verdict: PASS / FAIL, plus 2-3 specific reasons.
Use this prompt in a separate chat for each article.
Prompt, Solution B (if you need a batch):
You are an editor reviewing candidates' articles.
IMPORTANT: Evaluate each item in strict isolation.
Earlier ratings in our conversation must not influence
the next ones. Before each rating, mentally "reset"
our chat history.
Criteria: structure, headline, readability, grammar.
Article #[number]:
[text]
Verdict: PASS / FAIL + 2-3 reasons.
Result:
In option A, the model rates each article on a clean context, with no accumulated "momentum" from previous answers. In option B, the explicit instruction partly reduces drift but doesn't eliminate it, especially for borderline texts. You'll see the model justify each rating on its own merits, without leaning on the pattern of earlier answers.
Why It Works (and Why the Problem Exists at All)
An LLM doesn't evaluate; it continues text. The model doesn't look at the item with "fresh eyes". It sees the whole conversation and generates the next answer as a continuation of that pattern. If the conversation reads "no, no, no, no", the probability of a "no" token goes up. That's not a bug; it's the foundation of how every language assistant works.
Ambiguous items are the main casualty. When the call is clear-cut ("this code is completely broken" or "this article is brilliant"), the model is highly confident and the history doesn't tip the scale. But when the item is borderline, that's exactly where the history "reaches in" and tilts the balance. The paradox: the cases that matter most to evaluate (borderline decisions) are the most vulnerable.
Negativity hits harder because of how models are trained. LLMs have a built-in "negativity bias" in binary judgments: models lean toward "no" by default (several independent studies confirm this). Negative history catches a tailwind from a tendency that's already there. Positive history swims against the current.
Control levers: - History length → doesn't matter after 5 messages. Even 5 ratings produce the full effect. Don't kid yourself with "it's only 3 items in one chat": that's already enough. - An explicit "evaluate independently" instruction → reduces the effect but doesn't remove it. Better than nothing. - Alternating the polarity of the history → balance the batch if you really must batch.
Prompt Template
For rating a single item in an isolated context:
You are a {expert role} evaluating {item type}.
This is the only item you are seeing.
Evaluate it strictly on its own content.
Evaluation criteria:
— {criterion 1}
— {criterion 2}
— {criterion 3}
{item to evaluate}
Output: {answer format: PASS/FAIL or YES/NO}
+ {number} specific reasons.
Placeholders:
- {expert role}: editor, investor, lawyer, HR specialist
- {item type}: article, resume, business proposal, post
- {criterion 1-3}: specific evaluation parameters
- {item}: the text or description itself
- {answer format}: pick the binary output you need
For batch rating (when a single chat is unavoidable):
You are a {expert role} evaluating a series of {item type}.
RULE: Evaluate each item independently.
Earlier ratings in our conversation do not affect the next ones.
Before each rating, do a mental reset.
Criteria:
— {criterion 1}
— {criterion 2}
Item: {item}
Verdict: {answer format} + reasoning.
🚀 Quick start: paste this into your chat:
Here is a template for rating items in isolation.
Adapt it to my task: {your task: what you're rating and by which criteria}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what exactly you're rating, by which criteria, and what output format you need, because the expert role and the criteria determine the accuracy and style of the answer. It will take the isolation pattern from the template and plug in your details.
Limitations
⚠️ Clear-cut cases aren't affected: If an item is obviously bad or obviously good, the effect is minimal. The problem lives in borderline calls. Don't panic if you're filtering out blatant spam in a single window.
⚠️ An explicit instruction isn't a cure-all: Adding "evaluate independently" beats nothing, but the study shows the bias still remains. The only real protection is a fresh context.
⚠️ Bigger models reduce the effect but don't eliminate it: GPT-5.2 and Claude Opus suffer less than cheap models, but every significant model shows the effect. Don't count on "smart models will handle it on their own".
⚠️ Some models react the opposite way: The Qwen3 family showed the reverse effect, a contrarian bias (they moved away from the history's polarity instead of following it). That's still bias, just in the other direction.
How They Studied It
The idea was simple: take identical test tasks and show them to a model under two conditions, a clean context and after a history packed with "approvals" or "rejections". The team prepared three domains: code review, content moderation, and rating how healthy dishes are. Each model answered the same 21 test cases both with "fresh eyes" and after a history of 5, 10, 20 or 50 previous ratings.
75,898 API calls to 11 models from 4 providers. This isn't a small experiment: the scale lets you say with confidence that the result isn't a fluke. Each condition was repeated 10 times at high temperature to capture the probabilistic shift, not just the "edge cases".
The most surprising result: the saturation curve turned out flat. The researchers expected that more context would mean more bias. But 5 previous ratings produced exactly the same effect as 50. The model "reads the pattern" almost instantly, and adding more examples after that changes nothing. This matters in practice: even a "short" batch of 5 items already creates the problem.
It also turned out that a balanced history (50/50) still pushes the model toward "no", because of the built-in negativity bias. In other words, a "neutral" batch with an equal number of approvals and rejections still isn't neutral.
Adaptations and Extrapolations
💡 Adaptation: screening resumes on LinkedIn / freelancer bids on Upwork
Instead of running every application through one chat, set up a template so you can quickly open a new conversation for each candidate. Save the system prompt in your notes and paste it into a new chat. Five seconds, and the context is clean.
You are an experienced recruiter at a tech company.
Evaluate ONE resume. Do not compare it with other candidates.
Position: {role title}
Requirements: {key requirements}
Resume:
[paste resume text]
Output: INVITE / DO NOT INVITE to interview
+ 3 specific reasons, citing the resume text.
🔧 Technique: a "confidence anchor" before rating → less drift
Ask the model to explicitly rate its own confidence first. This partly kicks in its internal calibration before the context history can "tip the scale".
Before rating:
1. Rate your confidence: how clear-cut is this item on a 1-10 scale?
2. If your confidence is below 7, state what exactly makes it borderline.
3. Now give your verdict: {format}
This doesn't remove the effect, but it makes borderline cases visible: you can see which ratings are "questionable" and worth rechecking in a fresh context.
🔧 Technique: periodic "resets" inside a batch
If you're rating a long list and can't spin up separate chats, insert an explicit "divider" between items:
---NEW EVALUATION. IGNORE THE HISTORY ABOVE.---
Next item:
[item]
It's weaker than a fresh context, but it creates a semantic break in the conversation history.
Resources
AMEL: Accumulated Message Effects on LLM Judgments
Author: Temkit Sid-Ali
Preprint, May 2026
Related work cited in the paper: - "Old Habits Die Hard": a geometric analysis of conversational inertia in LLMs - "Mitigating Conversational Inertia": strategies for fighting inertia in agentic systems - B-SCORE (Vo et al.): a method for detecting and correcting bias using the response history