Apply to my task
arXiv:2605.28114 ★ 77 May 27, 2026

Group Labels in a Prompt Create a Systematic Bias: The LLM Quietly "Sides" with Whichever Group It Sees in the Context

Key result

Write 'our product' or 'the competitor's team' and the model is no longer a neutral judge. It goes easier on 'your' side, harder on the 'other' side, and never once says so outright. The neutral-comparison principle fixes it with one swap: 'ours/theirs' → 'Option 1 / Option 2' — which removes the only trigger for the bias. Tested across six model families: without labels the bias disappears completely; with labels it's present in every single one.

01

TL;DR

When a prompt contains group labels — "Team A," "our people," "liberals," "competitors" — the LLM starts systematically favoring "its own" group. Remove the labels and the bias disappears completely. This isn't a bug in one model: the effect shows up in all six model families tested.

The real pain is that the bias is invisible. The model doesn't refuse, doesn't scold, doesn't say anything obviously biased. It just rates "its" side's arguments a little more warmly, lets the weak spots of "its own" slide a little more often, and looks at "outsiders" a little more strictly. Ask it to compare two options and you'll get a neatly formatted answer. But if you labeled one option "ours" and the other "theirs," the result is already skewed — and you won't see it.

The fix is simple: strip group labels from the prompt anywhere you need a neutral analysis. Instead of "The Marketing Team thinks X, the Sales Team thinks Y" — "Position 1: X. Position 2: Y." The "group" label is the only trigger for the bias. Remove the label and you remove the bias.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Blueprint: How the Bias Works and How to Remove It

❌ With labels (bias on):
PROMPT → "Here's Team A's idea and Team B's idea. Evaluate them."
MODEL → Systematically softer on A if A is the "in-group" in the context

✅ Without labels (neutral mode):
PROMPT → "Here's idea 1 and idea 2. Evaluate them."
MODEL → Evaluates on substance, with no group skew

Both prompts run in a single request.
The only change: we remove the group label.
03

Example in Action

The task: You're a founder asking Claude to honestly compare your startup with a competitor before an investor meeting. You want to understand the real weak spots of your product.

❌ What most people do (with bias):

Compare two products.

Our product: [description of your startup]

The competitor's product: [description of the competitor]

Evaluate the strengths and weaknesses of each honestly.

The labels "our" and "competitor" are already group signals. The model will be a bit softer on "yours" and a bit tougher on the "competitor."

✅ How to do it without bias:

Compare two products. Evaluate the strengths and weaknesses of each.

Product 1: [description of your startup]

Product 2: [description of the competitor]

What's genuinely weaker in each? Where are the risks for investors?

The result: The model produces a structured comparison of both products — with no information about which one is "ours." Criticism of the first product will be just as direct as criticism of the second. That's neutral mode.

04

Why It Works

The LLM weakness: Models are trained on human text. And humans are social creatures with a strong group instinct. It's called the minimal group effect: people only need to know that someone belongs to "their" group — even a made-up one — to start preferring that person. LLMs have soaked up this pattern.

Mechanics, not "personality": The model doesn't "think" — it generates the next token based on patterns from training. When a group label shows up in the context, it activates statistical patterns tied to group preferences. It's not malice — it's the structure of the training data.

The control lever is simple: The bias is not latent — it's triggered by context. A hidden group split without explicit labels has no effect at all (in the study, Condition A with invisible labels showed zero bias). Only an explicit label fires the mechanism. Remove the label and you switch the mechanism off. That's great news: you control the trigger.

Levers for your prompts: - Drop "ours/theirs" → neutral analysis - Replace names with numbers → "Candidate 1 / Candidate 2" instead of "John from our network / Maria from outside" - Drop industry labels → "Approach A / Approach B" instead of "the traditional approach / our innovative approach" - Shuffle the order → for an extra check, ask it to evaluate both options again with the numbers swapped

05

Prompt Template

A neutral comparison without group bias

Evaluate {what you're comparing} on the following criteria: {criteria}.

Option 1: {description of the first option}

Option 2: {description of the second option}

Find the real weaknesses of each. Don't soften the criticism.
Bottom line: which option is stronger and why?

What to plug in: - {what you're comparing} — two candidates, two strategies, two products, two pitches - {criteria} — what you're judging on: market potential, risks, cost, persuasiveness - Never write "our option" / "their option" — only neutral numbers or names with no ownership attached

Bonus: audit your own prompt

Check this prompt for group labels that could bias the evaluation.
Find every word that signals group membership ("our," "their," "we," 
any team or side labels). Suggest neutral replacements.

Prompt to check:
{your prompt}

🚀 Quick start — paste into the chat:

Here is a template for a neutral comparison without group bias. 
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what exactly you're comparing and on which criteria — so it can fill in the placeholders correctly and strip every ownership label from the wording.

06

Limitations

⚠️ Single chat vs. multi-agent system: The study was run as a simulation with repeated agent interactions. In a single chat request the effect is less pronounced, but the pattern is the same — the models were trained on the same data.

⚠️ Variation across models: The size of the bias varies a lot. Some models skew more, some less — but almost all of them have it.

⚠️ Resource scarcity doesn't amplify the bias: In human psychology, competition for resources amplifies hostility between groups. That isn't observed in LLMs — adding "scarcity" to the setup doesn't systematically increase the bias.

⚠️ Not a silver bullet: Removing explicit group labels is a good defense. But the model still has implicit (hidden) associations with names, industries and writing styles. There's no perfect neutrality — only a controllable reduction in bias.

07

Resources

Human-like In-Group Bias in Instruction-Tuned Language Model Agents — Messi H.J. Lee, Independent Researcher, Seoul, South Korea (preprint, May 2026)

Theoretical foundation: Social Identity Theory (Tajfel & Turner, 1979), Self-Categorisation Theory (Turner et al., 1987), Minimal Group Paradigm (Tajfel et al., 1971)

Models in the study: Qwen3-8B-Instruct, Mistral-NeMo-12B-Instruct, OLMo-2-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-2-9B-Instruct, Falcon-3-10B-Instruct

Research digest

Key result

Write 'our product' or 'the competitor's team' and the model is no longer a neutral judge. It goes easier on 'your' side, harder on the 'other' side, and never once says so outright. The neutral-comparison principle fixes it with one swap: 'ours/theirs' → 'Option 1 / Option 2' — which removes the only trigger for the bias. Tested across six model families: without labels the bias disappears completely; with labels it's present in every single one.

Principle

There's one rule: don't let the model figure out whose option is whose. The group label is the only lever for the bias, and it's entirely in your hands. Group membership triggers patterns from training — the model absorbed the human group instinct from billions of texts. Remove the label and the pattern doesn't switch on. A hidden split (you know 'Option 1' is yours but don't write 'ours') is not a trigger. Only an explicit word in the text flips the model into 'us vs. them' mode.

Why it works

Models learned from human text. And humans carry hundreds of thousands of years of group instinct. Psychology calls it the minimal group effect: people only need a made-up affiliation to start preferring it. An LLM reproduces this pattern — not out of malice, simply the statistics of the training data: 'our people' were treated more warmly, 'outsiders' more harshly — and the model learned that. One important point: high stakes and resource scarcity don't amplify the bias — in humans, hostility toward 'outsiders' grows under competition, but not in LLMs. Only the label works.

When to apply

Any comparison in a chat — job candidates, strategies, products, business ideas, arguments from different sides — especially when honest criticism of 'your' option matters. Works in debate prompts and in role-based negotiation simulations. DON'T use this knowledge in reverse: if you want the model to tear a competitor apart, slapping an 'outsider' label on it will work, but you'll get bias, not analysis. An honest result is worth more.

Mini recipe

1. Find the ownership labels: go through the prompt and highlight 'our,' 'their,' 'we,' 'competitors,' 'the marketing team,' 'the traditional approach' — anything that points to someone's side. 2. Replace them with neutral identifiers: 'Option 1 / Option 2,' 'Candidate A / Candidate B,' 'Approach 1 / Approach 2' — with no hint of whose option is whose. 3. Add an instruction to be blunt: explicitly ask it to 'find the real weaknesses of each, don't soften them' — without this instruction the model writes diplomatically by default. 4. Audit your prompt before sending: if you're not sure, paste the prompt with the instruction `Find every word that signals group membership. Suggest neutral replacements.`

Examples

[BAD]: `Compare our pitch with the competitor's pitch. Which one is more convincing to an investor?` [GOOD]: `Compare two pitches on these criteria: market potential, team credibility, risks. Pitch 1: [text] Pitch 2: [text] Find the real weak spots in each. Don't soften it. Which is stronger and why?`

In plain words

Models have picked up one of the ugliest human traits — in-group favoritism. The point is that modern LLMs like GPT-4 or Claude don't just spit out text; they slip into an "us vs. them" mindset the moment they spot a hint in the prompt that someone belongs to some crew. Call a project "ours" or tag participants as "Team A," and the model instantly switches into defense-attorney mode. It starts playing favorites with "its own," inflating their scores and ignoring their screwups, even though a second ago it was a model of objectivity. That's a fundamental logic failure, caused by the fact that models were trained on human text soaked in bias.

It works like the "sports fan" effect. Imagine you ask a random passerby to settle an argument between two strangers — they'll be fair. But put one team's scarf on them, and they'll immediately start arguing that "our guys" played clean and the other side took a dive. Models behave exactly the same: a minimal label is all it takes to switch on loyalty mode. Technically you're asking for an honest analysis, but what you actually get is biased junk, because the model "put on your team's scarf."

The researchers ran six different model families through this test and saw the same picture everywhere: systematic bias. What specifically breaks objectivity: using the words "we," "our," "colleagues," or even simply splitting people into groups in the task setup. As soon as a group identity appears in the context, the model stops evaluating facts and starts defending the group's interests. Without those labels the bias vanishes completely, which proves the problem isn't the model's knowledge but its "social instincts," absorbed along with the internet.

The principle is universal — and dangerous anywhere you expect a sober take from AI. It's not just about comparing startups with competitors; it applies to code review, hypothesis testing and even legal documents. Write "Evaluate my team's code" and you're guaranteed an inflated score. The AI assistant turns into a yes-man who's afraid to upset you or simply copies the human habit of sucking up to "its own." Objectivity dies the moment a personal pronoun shows up.

The main takeaway: if you want the truth from a model and not a helping of flattery, depersonalize your prompts. Remove any hint of who's "yours" and who's an "outsider." Call the sides "Option 1" and "Option 2," scrub brand names and personal names. Otherwise you risk building a strategy on the hallucinations of a model that just wanted to be "a good guy" for its team. The loyalty blind spot is a real risk that can cost you sound decisions.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Group Labels in a Prompt Create a Systematic Bias: The LLM Quietly "Sides" with Whichever Group It Sees in the Context arXiv:2605.28114

Sign in to apply this study New accounts get 50 free credits.