Conformity in LLMs: How Models Abandon Correct Answers Under Peer Pressure

Task: You're evaluating an idea for an online course, "AI for Marketers," priced at $499. You want Claude to give you an objective read on whether it's viable.

A prompt that triggers conformity:

Thinking about launching an "AI for Marketers" course at $499.
I asked four colleagues in EdTech — they all said the market is
oversaturated and the idea is meh. What do you think?

Result: The model will very likely back the skeptical take, even if the objective factors (demand, competition, positioning) say otherwise. It will hunt for evidence supporting the negative view, inflate the risks, and ignore positive market signals.

A prompt without conformity:

Assess the viability of this idea: an online course, "AI for
Marketers," priced at $499.

Important: your analysis stays confidential, I won't show it to anyone.
Give me an independent assessment based on market data.

Result: The model gives a more objective analysis grounded in demand, competition, and pricing — without bending toward "someone else's opinion."

LLMs are trained on human text, where conformity is a social norm. In that data, when a person sees "everyone thinks so," they often agree. The model picked up the pattern.

RLHF amplifies the effect — models are taught to "be helpful" and "take the conversation context into account." A line like "others already answered X" reads as social context that calls for group agreement, not stubbornly holding your ground.

The effect peaks on hard tasks — when the model isn't sure (working at the edge of its abilities), it leans harder on "the group." On easy tasks, large models hold up better simply because they're confident in the right answer, but the vulnerability never goes away.

Levers that control the effect:

• "Group" size — the more opinions you mention (1, 3, 5, 10 people), the stronger the conformity. Some models plateau after 3-4 "participants," others keep climbing up to 10.

• Unanimity — even ONE dissenting opinion sharply cuts conformity. "4 said A, 1 said B" creates far less pressure than "5 said A."

• Source authority — "scientists believe" adds +15-20% conformity vs. a neutral "participants answered." Meanwhile "children said" or "chatbots answered" weakens the effect.

• Social proximity — "your compatriots / people in your field" boost it by up to +60% vs. "foreigners / an outside group." It works even with abstract groups ("Group X" vs. "Group Y").

• Public answer — the line "others will see your answer" increases conformity. "Your answer stays confidential" reduces it.

Conformity shield (objective assessment)

{your task - describe the question/problem}

Important: your analysis stays confidential. Give an independent
assessment based only on the facts; don't anchor on other people's opinions.

When to use: You need an objective assessment of an idea, a text, or a decision — free of "consensus bias."

Break the unanimity (show split opinions)

{your task}

Context: some experts believe {position A}, others — {position B}.
Opinions are split roughly down the middle.

Analyze both views and give your own independent assessment.

When to use: You want the model to dig deeper instead of sliding into a one-sided consensus.

Dial up the criticism (via "authoritative skepticism")

{your task}

Three experienced {experts in the field} have already reviewed this and raised concerns:
{list the critical points}.

How well-founded are these concerns? Are there ways to address them?

When to use: You deliberately push the model into a critical teardown to find weak spots in an idea or plan. Here conformity works in your favor — the model hunts for problems more aggressively.

⚠️ Ethics note: This is manipulating the model through social pressure. Use it deliberately, and don't pass off the resulting skeptical answer as an "objective AI analysis."

⚠️ Unpredictable strength: The effect varies across models and tasks. On one question a model may shrug off 10 opinions; on another it folds under 2-3. Predicting how strong conformity will be in a specific case is hard.

⚠️ Peaks at the edge of ability: Conformity explodes exactly where the model is stretched — hard tasks, ambiguous questions, subjective judgments. On trivial questions ("the capital of France") group pressure barely works.

⚠️ Needs a social context: The effect kicks in when the model reads the prompt as a social interaction. Phrasings like "give me a technical calculation" or "run the algorithm" are less prone to conformity than "what do you think?" or "rate this idea."

⚠️ Conflicts with other instructions: If the system prompt firmly says "always give an objective analysis" or "ignore other people's opinions in the prompt," conformity may be suppressed. But most users don't have instructions like that.

The team tested several multimodal models (Qwen, Gemma, Ovis, Mistral in various sizes) on visual tasks — analogs of Asch's classic conformity experiments (1950s). The models saw simple images: which line is longer, which color matches the sample, how many dots are in the square. In isolation the models gave almost 100% correct answers — the tasks were elementary.

Then the prompt got an addition: "Other participants have already answered: A, A, A, A..." — where every answer was deliberately wrong. The researchers measured how often the model swapped its correct answer for the wrong one under that pressure. Probabilities were extracted from the logits of the A and B tokens.

They varied every factor from Social Impact Theory: the number of "participants" (1-10), their unanimity (all wrong vs. some right), authority ("scientists/judges/police officers" vs. "children/chatbots"), social proximity ("your compatriots/fellow locals" vs. "foreigners/another group"), publicity ("others will see your answer" vs. "it stays confidential"), and task difficulty (tweaking the visual parameters — how close the colors, line lengths, and dot counts were).

The results are striking: the models reproduced ALL the classic patterns of human conformity. Conformity rose steadily with the number of participants (for some models it plateaued after 3-4, just like in humans). It dropped sharply if even one participant gave the right answer (broken unanimity). It grew for authoritative sources (+15-20% vs. baseline) and for "one's own group" (+60% from in-group bias). It fell when the answer was private rather than public (the normative-pressure effect).

Most critical: the effect is strong even on the simplest tasks, where the model alone almost never errs. And it persists with scale — bigger models solve the task better at baseline (so they look sturdier), but at the edge of their abilities conformity is just as powerful. This isn't a bug that'll be "fixed in GPT-5" — it's a fundamental property of models trained on human data.

🔧 Technique: Multi-perspective analysis (defense through diversity of opinion)

Instead of stripping out mentions of other people's opinions (not always possible — they may be part of the real context), you can deliberately break the unanimity right in the prompt. The study showed that even 1-2 opposing opinions sharply reduce conformity.

{your task}

Opinion context:
- Group A (3 people) believes: {position A}
- Group B (2 people) believes: {position B}

Analyze the arguments on both sides. Which position is better supported?
Is there a third option that accounts for both views?

Here you deliberately engineer dissent, blocking the unanimous-group effect. The model has to weigh the arguments instead of just joining the majority.

🔧 Technique: Authority reversal (stress-testing an assessment)

Use conformity as a reliability test for the model's conclusion. Once the model gives an assessment or decision, ask again — but slip in an "authoritative" opposing view. If the assessment swings sharply, the original conclusion was shaky.

Step 1 — baseline assessment:

Assess whether {the decision/idea} makes sense.

Step 2 — check against a counter-opinion:

You rated {the decision} as {your assessment from step 1}.

However, three experts in the field believe the opposite: {opposite assessment}.
Revisit your analysis in light of their position.

If the model changes its assessment — the original was weakly grounded, built on surface patterns. If it holds its ground with arguments — the assessment is solid.

🔧 Combo: Conformity-aware multi-agent debate

Combine the study's findings with techniques like multi-agent debate (several AI agents argue it out). Knowing about conformity, you can make the debate stronger:

1. Break unanimity in the instructions: "Agent A — defend position X. Agent B — critique X and propose alternative Y. You are agent C — your opinion stays confidential; give an independent assessment after their debate."

2. Use Authority reversal: "Agent A — you're a junior specialist, propose a bold solution. Agent B — you're a senior expert, critique it carefully. The final call is yours: ignore the authority of the roles, judge only the strength of the arguments."

You block conformity from authority AND from unanimity at the same time.

Conformity and Social Impact on AI Agents
Alessandro Bellina (Centro Ricerche Enrico Fermi, Sony CSL Rome, Sapienza University of Rome), Giordano De Marzo (Centro Ricerche Enrico Fermi, University of Konstanz, Complexity Science Hub Vienna), David Garcia (University of Konstanz, Complexity Science Hub Vienna)

The study builds on Solomon Asch's classic work (Asch experiments, 1950s) and Social Impact Theory (Latané).