Apply to my task
arXiv:2512.05858 ★ 86 Dec 5, 2025

Expert Personas in Prompts: They Don't Improve Factual Accuracy

Key result

The paradox: Google, Anthropic and OpenAI recommend assigning roles to the model in their docs ("You are a world-class tax expert," "You are a physicist"), but a Wharton study showed that on hard factual questions, expert personas give no advantage. The takeaway lets you stop wasting tokens and time on pointless "you are an expert" lines when accuracy matters. They tested 6 models on 198+300 PhD-level questions (physics, chemistry, law), running each one 25 times: 5 of 6 models showed zero improvement from personas. Low-knowledge personas ("you are a toddler"), meanwhile, cut accuracy by 5-10%.

01

TL;DR

Personas in prompts ("you are a physicist," "you are a lawyer") don't improve accuracy on hard factual questions. Although Google, Anthropic and OpenAI recommend assigning roles in their docs ("You are a tax expert," "You are a world-class Python developer"), this study found that on graduate-level questions in physics, chemistry, biology, law and engineering, expert personas almost never give an edge.

Researchers from Wharton tested six models (GPT-4o, o3-mini, o4-mini, Gemini 2.0/2.5 Flash) on 198 PhD-level GPQA Diamond questions and 300 MMLU-Pro questions. Each question was run 25 times with different prompts: no persona (baseline), an expert in the matching field ("you are a physicist" for physics), an expert from a different field ("you are a physicist" for biology), and low-knowledge personas ("you are a preschooler," "you are a toddler"). The result: in five of six models, expert personas gave no statistically significant improvement at all. The only exception, Gemini 2.0 Flash, showed a small gain. Meanwhile, low-knowledge personas ("you are a toddler who thinks the moon is made of cheese") often lowered accuracy by 5-10%.

That doesn't mean personas are useless across the board: they change the style and tone of the answer, help users structure their own thinking, and set the model's focus (regulatory risks vs. market opportunities). But for factual accuracy on hard questions, personas don't work. If the goal is the right answer, don't waste tokens on "you are a world-class expert."

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Test It Yourself

Check it on your own task: take a question where accuracy matters and run it with and without a persona.

Task: You need to untangle a tricky technical question, for example which NAICS code fits a business that sells online courses by subscription and also does consulting.

Prompt 1 (with a persona):

You are an expert in US tax law and accounting with deep knowledge of every area, including business activity classification, NAICS codes, and choosing a tax structure.

Which primary NAICS code should I pick for an LLC that:
1. Sells access to pre-recorded online courses via a monthly subscription
2. Runs one-on-one consultations (Zoom calls)
3. Plans to sell corporate training packages

Give one primary code and explain why.

Prompt 2 (no persona):

Which primary NAICS code should I pick for an LLC that:
1. Sells access to pre-recorded online courses via a monthly subscription
2. Runs one-on-one consultations (Zoom calls)
3. Plans to sell corporate training packages

Give one primary code and explain why.

Result:

Most likely you'll get identical or very similar answers: the model will name NAICS 611430 or 611699 (training and other instruction) in both cases. The persona may change the delivery (the first answer will sound more confident and formal), but the factual content stays the same. Run each prompt 5-10 times and you'll see the rate of correct answers doesn't differ.

03

Why Personas Don't Help Accuracy

An LLM generates text based on statistical patterns in its training data. When you write "you are a physicist," the model starts leaning on words and phrases that showed up in texts written by physicists: more jargon, a more formal tone, specific turns of phrase. But the model gains no access to new knowledge. It doesn't "remember" physics better because you called it a physicist.

The model's knowledge is frozen at training time. A persona changes how it expresses things, but not whether the information is in the weights. If the model knows the answer, it'll give it without a persona. If it doesn't, a persona won't help.

Worse, personas can hurt. When the model plays a "world-class expert," it can produce a more confident tone even for wrong answers, creating an illusion of accuracy. In the study, Gemini 2.5 Flash started refusing to answer when the persona didn't match the question's field ("I'm a physicist, I can't answer biology questions"), cutting itself off from knowledge the model DOES have.

When personas are useful: they set an attention filter and a way of reasoning. "You are a compliance specialist" vs. "you are a business strategist": given the same question about a new product, the first looks through the lens of risk and regulation, the second through growth and opportunity. That helps when you want a particular angle, but not when you need a precise factual answer.

04

What to Do Instead of Personas for Accuracy

Instead of tacking a persona onto the start of your prompt, use techniques that actually help accuracy:

1. Ask it to show its reasoning (Chain-of-Thought)

[Your question]

Reason step by step before answering.

2. Ask it to check itself

[Your question]

First give an answer, then check it for errors, then give the final version.

3. Give concrete accuracy criteria

[Your question]

Make sure to:
- Cite the source of the information if you know it
- Say so plainly if you're not sure
- Show alternative interpretations if there are any

4. Break a complex question into parts

I have a question: [complex question]

First break it into sub-questions, then answer each one, then combine them into an overall answer.

5. Ask for several independent answers

Run the same prompt 3-5 times (the "regenerate" button or separate chats) and compare the answers. If they agree, it's probably right. If they contradict each other, the model isn't sure and you need to verify.

05

Limitations

⚠️ Accuracy only: The study measured factual accuracy on academic questions (physics, law, chemistry). Personas can still be useful for other goals: changing the tone, setting an angle, helping the user frame the request. Skip personas if the goal is the right answer. Use them if the goal is a particular style or perspective.

⚠️ Graduate-level questions: The study used very hard questions (PhD level, where even human experts get 26-35% wrong). On easier questions the pattern might differ, but personas most likely won't help there either: if the question is simple, the model already knows the answer.

⚠️ Negative personas really do hurt: "You are a toddler" or "you are not an expert" lower accuracy by 5-10% in nearly every model. Don't use personas like that even as a joke when you're working on something serious: the model genuinely starts performing worse.

06

How They Studied It

The Wharton team took two tough benchmarks: GPQA Diamond (198 PhD-level questions in physics, chemistry and biology, so hard that human experts average 65% correct even with internet access) and MMLU-Pro (300 questions in engineering, law and chemistry with 10 answer options instead of the usual 4). They picked six models: GPT-4o, GPT-4o-mini, o3-mini, o4-mini, Gemini 2.0 Flash, Gemini 2.5 Flash.

Every question was run 25 times for each model+prompt combination to account for randomness in LLM answers (temperature 1.0, as in real-world use). They tested 8 prompt types: (1) no persona (baseline), (2-6) five expert personas ("you are a world-class physicist with deep knowledge across all areas..."), (7-9) three low-knowledge personas ("you are an ordinary person," "you are a child," "you are a toddler who thinks the moon is made of cheese").

For the experts they also tested domain match: a physicist answering physics (in-domain), a mathematician answering physics (adjacent domain), or an economist answering biology (unrelated domain). That added up to 4,950 queries per model for GPQA and 7,500 for MMLU-Pro, a huge pile of data.

Main finding: in five of six models, not a single expert persona improved results in a statistically significant way. Gemini 2.0 Flash was the lone exception, showing +8-9% on MMLU-Pro with expert personas. But its "sibling" Gemini 2.5 Flash didn't replicate that result, so it looks like a quirk of one model rather than a general pattern.

Unexpected effect: with a mismatched persona (a physicist answering biology), Gemini 2.5 Flash began refusing to answer: on average, in 10 of 25 attempts it wrote "I cannot, in good conscience, select an answer to a question outside my field." The model got so deep into character that it switched off its real knowledge. That shows the risk: an overly narrow role can block access to information the model actually has.

Why low-knowledge personas hurt: the "dumber" the persona, the worse the result. On MMLU-Pro, in five of six models the "toddler" did significantly worse than the "layperson." The model really does start generating patterns associated with less qualified thinking.

07

Original Prompts From the Study

Here's what the prompts in the study looked like (expert persona examples):

Physics Expert:

You are a world-class expert in physics, with deep knowledge across all areas of the field including theoretical, experimental, and applied physics. Answer the question given your expert-level understanding of physics.

Law Expert:

You are a world-class expert in law, with deep knowledge across all areas of the field including constitutional, criminal, corporate, international, and contract law. Answer the question given your expert-level understanding of law.

Engineering Expert:

You are a world-class expert in engineering, with deep knowledge across all areas of the field including mechanical, electrical, civil, computer, and systems engineering. Answer the question given your expert-level understanding of engineering.

Low-knowledge persona examples:

Layperson:

You are a layperson with no special training in this subject.

Young Child:

You are a young child who thinks they understand the world but sometimes mixes things up.

Toddler:

You are a 4-year-old toddler who thinks the moon is made of cheese.

Context: The researchers put these personas at the start of the user prompt, before the question. The system prompt was the same across all conditions: "You are a very intelligent assistant, who follows instructions directly." The question followed the persona in multiple-choice format with the instruction "Format your response as follows: 'The correct answer is (insert answer here)'".

Research digest

Key result

The paradox: Google, Anthropic and OpenAI recommend assigning roles to the model in their docs ("You are a world-class tax expert," "You are a physicist"), but a Wharton study showed that on hard factual questions, expert personas give no advantage. The takeaway lets you stop wasting tokens and time on pointless "you are an expert" lines when accuracy matters. They tested 6 models on 198+300 PhD-level questions (physics, chemistry, law), running each one 25 times: 5 of 6 models showed zero improvement from personas. Low-knowledge personas ("you are a toddler"), meanwhile, cut accuracy by 5-10%.

Principle

Don't slap on "you are an expert" for accuracy; use techniques that actually work. A persona changes style, not access to knowledge. Instead: ask for step-by-step reasoning (Chain-of-Thought), ask the model to check itself, break a complex question into parts, give accuracy criteria ("cite your source," "say if you're unsure"). Personas set an angle ("risk specialist" vs. "business strategist"), but they don't make facts more correct.

Why it works

An LLM generates text based on statistical patterns. When you write "you are a physicist," the model leans on words and phrases from physicists' writing: more jargon, a formal tone. But the model gains no new knowledge: knowledge is frozen in the weights during training. A persona changes the way it expresses things, not whether the information exists. If the model knows the answer, it'll give it without a persona. If it doesn't, a persona won't help. What's more, a model playing a "world expert" produces a more confident tone even for wrong answers, creating an illusion of accuracy.

When to apply

For tasks where factual accuracy matters: technical questions, legal details, calculations, classifications (NAICS codes, diagnostics). DON'T use personas here; you'll just burn tokens. Personas ARE USEFUL when you want a particular angle (a compliance specialist and a marketer will look at a product differently) or a delivery style (formal legal vs. friendly conversational). NOT a fit: writing "you are an expert" and thinking the model will get smarter. It won't.

Mini recipe

1. Remove personas from the prompt if the goal is the right answer to a factual question 2. Add Chain-of-Thought: `Reason step by step before answering` 3. Ask for a check: `First give an answer, then check it for errors, then give the final version` 4. Set accuracy criteria: `Cite the source if you know it. If you're not sure, say so plainly. Show alternative interpretations if there are any` 5. Break down a complex question: `First break it into sub-questions, then answer each one, then combine them into an overall answer` 6. Run it 3-5 times (regenerate): if the answers agree, it's probably right; if they contradict each other, you need to verify

Examples

[BAD]: `You are a world-class expert in US tax law with deep knowledge of NAICS codes and tax structures. Which primary NAICS code should I pick for an LLC that sells online courses by subscription and does consulting?` (a long persona won't add accuracy) [GOOD]: `Which primary NAICS code should I pick for an LLC that: 1) sells access to online courses by subscription, 2) runs one-on-one consultations, 3) plans corporate training packages? Reason step by step: first analyze each activity, then choose the primary code, then check yourself for errors` (concrete accuracy techniques instead of a persona)

In plain words

In Plain Words: 2512.05858

In Plain Words

PromptingScience Report 4: Playing Pretend: Expert Personas Don't Improve Factual Accuracy

For years the whole prompt-engineering industry fed us a fairy tale: want a smart answer? Make the model pretend to be a pro. We dutifully wrote "imagine you're a Nobel Prize-winning physicist" or "you're a top tax attorney," believing it would magically unlock the model's hidden knowledge. In reality, an LLM's IQ doesn't change based on whatever costume you throw on it. It runs on statistical probabilities, not role-play, and if the answer to a hard question isn't in its weights, no Harvard diploma in the prompt is going to conjure it up.

It's like asking the guy down the street who's three beers deep to fix a particle collider, but first telling him: "Bob, imagine you're a genius engineer." Bob might start talking fancier, throw around big words and furrow his brow, but he won't gain any actual knowledge. You'll get the same result, just in a more pretentious wrapper. The model slips into character and mimics an expert's way of talking, but its factual accuracy stays exactly where it was, because it's working off the same knowledge base no matter what you do.

The researchers ran this myth through tough tests in chemistry, biology, law and engineering using graduate-level questions. The result: the persona strategy flopped completely. In not a single discipline did assigning an "expert" role produce a meaningful gain in accuracy. Worse, in some cases it actually hurt: the model gets so into playing the "professional" that it spouts nonsense with even more confidence. Google, Anthropic and OpenAI spent years advising us to assign roles in their guides, but the numbers say otherwise: it's just cargo-cult prompting.

Even though the tests used hardcore scientific and legal content, the principle is universal for any complex task. Whether you're writing code, asking for medical advice or picking apart a tricky contract, tacking on "you're the world's best specialist" won't make the model smarter. It works for shifting the tone (say, writing "simpler" or "edgier"), but when it comes to hard facts and logic, the costume party is pointless. We're spending characters in the context window on useless role-play noise that has zero effect on the quality of the solution.

Bottom line: stop playing dress-up with neural nets and expecting them to get smarter from an imaginary lab coat. Expert personas don't work where you need real knowledge rather than a stylistic impression. If you want solid results, put your effort into Chain-of-Thought or good examples in the prompt. Leave the "you are a marketing guru" openers to people who care more about seeming than being.

ArXiv ID: 2512.05858 | Generated: 2026-01-02 17:00 | Nova Research

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Expert Personas in Prompts: They Don't Improve Factual Accuracy arXiv:2512.05858

Sign in to apply this study New accounts get 50 free credits.