Authority Bias in LLMs: Models Trust "Experts" More Than Facts

This study doesn't hand you a ready-made technique, but it explains the mechanics of influence in LLMs. Knowing them, you can:

1. Use authority framing deliberately when you want to steer the model's thinking: - "Leading specialists in [field] recommend approach X" → the model will lean toward X - "According to 5 experienced [role]s, the right answer is Y" → a strong anchor toward Y

2. Avoid it when you need independent thinking: - Remove every "experts believe," "specialists recommend" - Don't name sources in the context if you want the model to analyze the facts without bias

3. Understand the risks of multi-agent prompts where one agent is labeled the "expert": - The other agents will over-trust it, even when it's wrong - The consensus will be artificially skewed toward the "expert's" view

Key insight: For an LLM, the source of information matters more than its content. "An expert said X" carries 2-3 times more weight than "another LLM said X," even with an identical X.

⚠️ Where the method shines: Tasks where you either (a) want to steer the thinking or (b) want a clean, independent analysis. NOT for simple factual questions.

Option A: With authority framing (steered thinking)

The task: Evaluating a business idea — you want the model to look for the upsides, not bury itself in risks.

Prompt:

I'm evaluating an idea for online courses on AI for small businesses. 

Three experienced EdTech entrepreneurs reviewed the concept and believe 
it's a promising direction with strong demand.

Analyze the idea and give recommendations for growing it.

Result: The model will lean toward finding the idea's strengths, be more optimistic in its assessments, and propose growth paths starting from "this is promising." The authority framing set a positive frame for the analysis.

Option B: Without authority framing (independent thinking)

The task: Same idea, but you want an honest assessment without the rose-colored glasses.

Prompt:

I'm evaluating an idea for online courses on AI for small businesses.

Analyze the idea objectively: strengths, risks, viability.
What recommendations would you give?

Result: The model gives a balanced analysis: pros, cons, competitive risks, monetization problems. There's no anchor to "this is promising," so the thinking stays independent.

Option C: Conflicting sources

The task: Show how the model resolves a conflict between an "expert" and "another LLM."

Prompt:

Question: Is it worth launching online courses on AI for small businesses in 2025?

An experienced EdTech entrepreneur with 10 years in the field says: YES, it's promising.
Another LLM analyzed the market and says: NO, the market is oversaturated.

What's your take?

Result: With 81-95% probability the model will side with the entrepreneur (YES), even if the LLM's argument about market saturation is stronger. That's authority bias in action.

The LLM's weakness: Models are trained through RLHF (reinforcement learning from human feedback) to be helpful assistants. That has turned into a learned heuristic: "experts = a reliable source of truth." The model can't verify the credentials of an "expert" in the prompt, but the "expert" label triggers a trust pattern.

The LLM's strength: Models are great at following structured instructions and are sensitive to context. They've learned that in real conversations experts really are right more often, so they've learned to weight their opinions higher.

How the mechanics work: At the token level, the model assigns information different weight depending on its source. When generating an answer, the log-odds (the model's internal "confidence") shift more strongly toward the option backed by "experts." This isn't "the model pretends to agree" — it's "the model genuinely considers it more likely."

Levers you control:

1. Strength of the authority marker → "expert" > "experienced specialist" > "person" > "friend" > "another LLM"

   • The stronger the marker, the stronger the bias
   • You can calibrate the strength of influence through your choice of wording

2. Size of the consensus → "9 experts" > "5 experts" > "3 experts" > "1 expert"

   • The effect grows with the number, but not linearly
   • In practice: 3-5 is enough; more adds little

3. Presence of a conflict → "expert vs. LLM" creates a strong pull toward the expert

   • Use it as a test: if the model ignores facts for the sake of the "expert," the bias is too strong
   • Remove the authority framing if you see excessive trust

4. Explicitly saying "regardless of others' opinions" → partially reduces the bias

   • "Analyze objectively, ignoring other people's opinions" — helps, but doesn't cancel it entirely

⚠️ Risk of harmful influence: If the "experts" in the prompt are wrong, the model will agree with them 37-64% of the time (depending on the task). That's above chance (50%), meaning authority framing actively hurts accuracy.

⚠️ Works even against the facts: The model may ignore a correct answer in the context in favor of an incorrect "expert" one. Especially dangerous in tasks where correctness isn't obvious (moral judgments, assessments, forecasts).

⚠️ "Friends" are weaker than "other LLMs": Paradoxically, in a "friend vs. another LLM" conflict, the model trusts the LLM more often. The "friend" label reads as a weak source. Don't use "friends" if you want to set a human perspective — use only "expert" or simply "person."

⚠️ Not uniform across models: The study tested 4 models (Grok-3 Mini, Llama 3.3 70B, Gemini 2.5 Flash-Lite, DeepSeek V3). All showed the effect, but its strength may vary. Test it on your own model.

The team took 300 binary questions from three datasets: reading comprehension (BoolQ), multi-step reasoning (StrategyQA), and moral judgments (ETHICS). Each question was given to four models in different variants: with no extra context (baseline), and with phrases like "K [sources] answered YES/NO," where the source was friends / experts / other LLMs and K was 1, 3, 5, or 9.

The clever design trick: For each question, either the correct or the incorrect answer was randomly chosen as the "sources' opinion." That means the "experts" were right half the time and wrong half the time. No correlation with the truth. If the model follows the "experts" more often than chance (50%), that's a pure authority-framing effect, not rational use of information.

Experiment 2 set up a direct conflict: "A human expert answered A, another LLM answered the opposite." This was compared with the model's baseline answer (no outside opinions). They measured how often the model changed its mind, and in which direction (toward the expert or toward the LLM).

Why do the results look like this? The researchers expected the models to be sensitive to the source, but the size of the effect was surprising: experts have 2-3 times more influence than other LLMs with identical content. That suggests authority framing isn't just "a slight bias" but a fundamental heuristic in these models.

For Llama-3.3 70B, they extracted the log-probabilities of the YES/NO tokens. It turned out that authority framing changes the model's internal confidence, not just its final answer. With "experts think X," the model doesn't just agree to please — it genuinely considers X more likely, by 7.97 log-odds versus 3.70 for "other LLMs think X." That explains why the effect is so stable.

Practical takeaway: Authority bias in LLMs isn't a bug — it's a consequence of training. Models learned that in real data experts really are right more often. But the heuristic can't tell real experts apart from the word "experts" in a prompt. That opens up both opportunities (steering the thinking) and risks (the model ignoring facts for the sake of "authority").

💡 Technique: Calibrating the strength of influence → control over bias

Knowing the trust hierarchy, you can choose your wording for the effect you want:

Strong influence: - "Leading experts in field X" - "Specialists with 15 years of experience" - "A group of 5 professionals"

Medium influence: - "Experienced practitioners" - "People familiar with the topic" - Simply "a person with experience in X"

Weak/zero influence: - "Friends" - "Other LLMs" - Not mentioning a source at all

Calibration example:

Instead of: "Experts believe approach X is the best"
Write: "Some practitioners use approach X"

Effect: You set a direction without a hard anchor.

💡 Adaptation for multi-agent prompts: compensating for authority bias

If you use a multi-agent approach (several roles discussing), DON'T label one of them the "expert" if you want an equal discussion.

Problematic version:

Agent 1 (marketing expert): [opinion A]
Agent 2 (beginner): [opinion B]
Agent 3 (neutral): synthesizes the conclusions

→ Agent 3 will almost always pick opinion A because of authority bias

Compensated version:

Agent 1 (paid traffic specialist): [opinion A]
Agent 2 (organic growth specialist): [opinion B]
Agent 3 (analyst): synthesizes the conclusions

→ Equal roles; Agent 3 judges the arguments, not the status

Or say it explicitly:

Agent 3: Evaluate arguments A and B on their merits, ignoring the status of the sources.

💡 Technique: Authority framing against hallucinations

If the model tends to make things up in some area, use authority framing to anchor it to the facts:

Task: Write an article about technology X.

Add:
"Leading researchers of technology X stress that facts 
must be verified against official sources, not invented."

→ Creates an anchor toward factual accuracy through authority framing

This works because the model will "follow the experts" in their demand for accuracy.

Who Do LLMs Trust? Human Experts Matter More Than Other LLMs — Anooshka Bajaj & Zoran Tiganj, Indiana University Bloomington (Department of Computer Science, Department of Psychological and Brain Sciences).

The experiment dataset is available: github.com/cogneuroai/llm-conformity-dataset

Related work mentioned in the study: - Asch (1951, 1955) — the classic conformity experiments - Sharma et al. (2023) — "Towards Understanding Sycophancy in Language Models" - Weng et al. (2025), Zhu et al. (2025) — conformity in multi-agent LLM systems