Apply to my task
arXiv:2511.20086 ★ 76 Nov 25, 2025

BiasPrompting: Forcing the Model to Explore Every Option Before It Chooses

Key result

Finding: LLMs "lock onto" the first plausible option: they build reasoning for it alone and drop the rest without analysis. BiasPrompting lets you make well-weighed decisions when choosing among several alternatives: the model explores every option, not just the first one it stumbles on. The trick: make the model find arguments FOR every option, even an absurd one. The model defends each as if it were definitely right. Then, with all the arguments in front of it, it picks the most convincing.

BiasPrompting: Forcing the Model to Explore Every Option Before It Chooses

BiasPrompting: Forcing the Model to Explore Every Option Before It Chooses

02

TL;DR

BiasPrompting is a two-step technique for multiple-choice tasks. First the model generates arguments in favor of each option (as if each one were correct); then, with all the arguments in front of it, it picks the most convincing one.

When an LLM sees a question with answer choices, it often "locks onto" the first plausible one: it builds reasoning for that option alone and drops the rest without analysis. This is made worse by selection bias: models tend to prefer certain positions (say, the first or last option) regardless of content. As a result, strong alternatives go unexamined.

BiasPrompting fixes this with forced exploration: the model must find arguments for every option, even an absurd one. That gives it the full picture before the final choice, like a judge who has heard every side's case before ruling.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

03

Method Blueprint

STEP 1: Generate arguments
   For each option A, B, C... → "Why is [option] the correct answer?" → argument

STEP 2: Choose based on all the arguments
   Question + all options + all arguments → "Which argument is most convincing?" → final answer

Both steps can run in a single prompt or as two separate requests (to quality-check the arguments).

04

Example in Practice

Task: You're choosing a CRM for a small real estate agency. There are three options: HubSpot, Salesforce, Pipedrive. You need to make an informed decision.

Prompt:

I'm choosing a CRM for a real estate agency (5 agents, ~50 deals/month).

Options:
A. HubSpot
B. Salesforce
C. Pipedrive

STEP 1: For each option, write 2-3 sentences on why IT, specifically, is the best choice for my situation. Defend each option as if it were definitely the right one.

STEP 2: Look at all the arguments. Which option is most convincing for a real estate agency with 5 agents? Explain why the arguments for it outweighed the rest.

Result: The model will produce a structured answer: first a block of arguments for HubSpot (free tier, marketing tools, Zillow/Realtor.com lead integrations), then for Salesforce (all-in-one, deep customization), then for Pipedrive (dead-simple sales pipeline). At the end, a weighed choice explaining which arguments proved stronger for this specific situation.

05

Why It Works

The LLM's weakness: Models are prone to confirmation bias: once they find a plausible option, they build their reasoning around it and ignore the alternatives. On top of that there's selection bias: an option's position in the list affects the choice regardless of content. Swap options A and C and the model may pick a different answer.

The LLM's strength: Models are great at generating arguments in a given direction. Tell one to "defend position X" and it will find relevant facts and logic. That can be used to systematically explore every option.

How the method uses this: BiasPrompting turns the weakness into a strength: instead of one biased line of reasoning, the model creates a set of biased lines of reasoning (one for each option). With all the arguments gathered together, the model sees the whole picture and can make a comparative choice rather than grabbing "the first plausible one".

Control knobs:

  • Depth of arguments — "2-3 sentences" vs. "a full paragraph of analysis" → more detail for complex decisions
  • Evaluation criteria — add specific criteria to STEP 2 ("on price", "on simplicity", "on scalability") → focus on what matters to you
  • Output format — ask for a comparison table after the arguments → visual structure for the decision
06

Prompt Template

{situation_context}

Options:
A. {option_1}
B. {option_2}
C. {option_3}

STEP 1: For each option, write arguments for why IT, specifically, is the best choice. Defend each option as if it were definitely the right one.

STEP 2: Look at all the arguments. Which option is most convincing? Explain why its arguments outweighed the rest.

Placeholders:

  • {situation_context} — what you're choosing and why (the more specific, the sharper the arguments)
  • {option_N} — the options (2 to 6 works well)
07

Limitations

⚠️ Argument quality: If the model doesn't know the subject well, the arguments will be shallow or wrong. For specialized topics (medicine, law, niche tech), give the model context first or use web search.

⚠️ Not for simple questions: If the answer is obvious ("Capital of France?"), the method is overkill: the model will burn tokens generating pointless arguments for Lyon and Marseille.

⚠️ Subjective criteria: The method works better when there are objective evaluation criteria. For matters of taste ("which color is better?"), the arguments will be equally convincing.

08

How They Tested It

A team from NTU (Singapore), NUS, and VinUniversity (Vietnam) tested BiasPrompting on five benchmarks of commonsense questions: CommonsenseQA, StrategyQA, PIQA, and two from BIG-Bench (Date Understanding, Causal Judgement). They used three 7B models: Mistral, DeepSeek, Gemma.

An interesting finding: CoT (Chain-of-Thought) is unstable: +11% on some datasets, −14% on others. BiasPrompting showed more consistent gains almost everywhere. And BiasPrompting generates fewer tokens than CoT, because it doesn't require long reasoning chains.

They also tested robustness to option order: the answer choices were shuffled randomly and the spread of results measured. BiasPrompting showed lower variance: the model "locked onto" a particular position less often. That confirms that forced exploration of all options really does reduce selection bias.

One more experiment: they looked at questions BiasPrompting gets right but zero-shot and CoT get wrong. There turned out to be significantly more of those than the reverse. This suggests the method "unlocks" hidden abilities: the model knew the right answer but couldn't reach it without structured exploration of the alternatives.

09

Original Prompts from the Paper

Context: Prompt template for generating arguments (STEP 1): the model gets the question and one option and has to defend it as correct.

Given the following question:
A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
Provide reasoning proving that 'bank' is the correct choice without any textual description in one sentence.

Context: Prompt template for the final choice (STEP 2): the model sees all the arguments and chooses.

### Question: A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
### Answer choices: A. bank | B. library | C. department store | D. mall | E. new york
### Reasoning for answer choice A: The answer is 'bank'. Revolving doors are often used in banks to control the flow of people entering and exiting.
### Reasoning for answer choice B: The answer is 'library'. Revolving doors are commonly used in libraries to control the flow of people entering and exiting.
### Reasoning for answer choice C: The answer is department store. Revolving doors are commonly used in department stores to control the flow of customers.
### Reasoning for answer choice D: The answer is 'mall'. Revolving doors are commonly found in malls to control pedestrian traffic flow and prevent congestion.
### Reasoning for answer choice E: The answer is New York. The revolving door is a security measure at the Empire State Building in New York City.

Wrap your final answer by filling in the placeholder below: 'So the answer is: {{placeholder}}'
10

Adaptations and Extrapolations

💡 Adaptation for career decisions

BiasPrompting works well for weighing career alternatives, where emotions often get in the way of objective analysis.

I'm 28 and have spent 4 years as a product manager in fintech. I've got three offers:

A. Stay at my current company (promotion to Senior PM, +30% salary)
B. Move to Google (same level, but bigger scale, salary on par)
C. Join a startup as a co-founder (lower salary, but 5% equity)

STEP 1: For each option, write arguments for why it's the best choice for a product manager's career over a 5-year horizon. Defend each option persuasively.

STEP 2: Which path offers the best combination of skill growth, finances, and risk? Why did the arguments for it win out?

🔧 Technique: add evaluation criteria → a focused comparison

If you add explicit criteria to STEP 2, the model will compare the arguments against them instead of in the abstract.

STEP 2: Evaluate the arguments on three criteria:
- Salary growth potential over 3 years
- Hard skills development
- Work-life balance

Which option wins on the combined criteria?

💡 Extrapolation: BiasPrompting + Devil's Advocate

You can strengthen the method by adding a third step: a critique of the chosen option. This helps catch blind spots.

STEP 1: [arguments for each option]

STEP 2: [choose the best]

STEP 3: Now play devil's advocate: what are the 3 biggest risks of the chosen option? What could go wrong?
11

Resources

Paper: "More Bias, Less Bias: BiasPrompting for Enhanced Multiple-Choice Question Answering" (SAC'26)

Authors: Duc Anh Vu, Nguyen Thanh Thong, Cong-Duy Nguyen, Viet Anh Nguyen, Anh Tuan Luu

Organizations: Nanyang Technological University (Singapore), National University of Singapore, VinUniversity (Vietnam)

Related work: Selection bias in LLMs (Zheng et al., 2023), Chain-of-Thought prompting (Wei et al., 2022), Self-Consistency (Wang et al., 2022)

Research digest

Key result

Finding: LLMs "lock onto" the first plausible option: they build reasoning for it alone and drop the rest without analysis. BiasPrompting lets you make well-weighed decisions when choosing among several alternatives: the model explores every option, not just the first one it stumbles on. The trick: make the model find arguments FOR every option, even an absurd one. The model defends each as if it were definitely right. Then, with all the arguments in front of it, it picks the most convincing.

Principle

A two-step process. Step 1: For each option A, B, C: "Why is IT, specifically, correct?" → generate arguments. Step 2: All the arguments are gathered → "Which is most convincing?" → final choice. The model stops grabbing the first plausible option and starts comparing them all on equal terms. Works in one prompt or as two requests (to quality-check the arguments).

Why it works

LLMs are prone to confirmation bias: once they find a plausible option, they build their reasoning around it and ignore the rest. Plus selection bias: position in the list affects the choice regardless of content (swap A and C and the model may pick a different answer). BiasPrompting turns the weakness into a strength: instead of one biased line of reasoning, the model creates a set of biased lines of reasoning, one for each option. With all the arguments gathered together, the model makes a comparative choice instead of grabbing "the first plausible one".

When to apply

For decisions with several options to choose from (2-6 alternatives): picking a tool, an approach, a strategy, especially when you need a well-grounded decision that accounts for all the pros and cons. NOT a fit for obvious questions (capital of France?) or purely subjective preferences with no objective criteria (which color is better?).

Mini recipe

1. Describe the decision context: what you're choosing and why (the more specific, the sharper the arguments) 2. List the options: A, B, C... (2 to 6 works well) 3. STEP 1 in the prompt: `For each option, write 2-3 sentences on why IT, specifically, is the best choice. Defend each one as if it were definitely the right one.` 4. STEP 2 in the prompt: `Look at all the arguments. Which option is most convincing? Explain why its arguments outweighed the rest.` 5. Optional: add evaluation criteria to STEP 2 — `on price`, `on simplicity`, `on scalability`

Examples

[BAD]: `Which CRM should I pick for a real estate agency — HubSpot, Salesforce, or Pipedrive?` (the model will immediately pick the first familiar option without deep analysis) [GOOD]: `I'm choosing a CRM for an agency (5 agents, ~50 deals/month). Options: A. HubSpot B. Salesforce C. Pipedrive. STEP 1: For each option, write 2-3 sentences on why IT, specifically, is best for my situation. Defend each one as if it were definitely the right one. STEP 2: Look at all the arguments. Which option is most convincing for an agency with 5 agents? Why did its arguments win out?` (the model will generate arguments for HubSpot's free tier and marketing tools, Salesforce's all-in-one depth, and Pipedrive's simple pipeline, then make a weighed choice)

In plain words

On multiple-choice tests, models behave like overconfident amateurs: they latch onto the first option that catches their eye and start bending the facts to fit it. That's called confirmation bias. If the model has decided the right answer is A, it'll ignore every argument against it. Worse, even the order of the words affects the choice: put the correct answer at the end of the list and the model skips it simply because it got tired of reading. The BiasPrompting method breaks this crooked logic by making the AI work systematically.

It's as if you were shopping for a car and, instead of immediately buying "that cute red one", you hired three different salespeople. One argues that BMW is the best, the second goes all in on Mercedes, and the third pitches a scooter. You hear out each of them as they bend over backwards to justify their pick, and only then, with all the arguments in hand, do you decide. Technically you're making the model lie for a good cause, forcing it to find upsides even in an answer that's obviously wrong.

Technically it runs in two stages. First comes argument generation, where for each option (A, B, C) the model writes a case for why that one is correct. It literally writes three different essays titled "Why I'm Right". Then comes the final choice stage, where the AI steps back, looks at its own arguments, and judges which one sounds most logical. The method kills position bias, where the model picks an option just because it's first in the list.

Although the method was tested on dull multiple-choice tests, the BiasPrompting principle is a lifesaver for any tough choice, from buying a CRM to hiring an employee. Instead of asking the model "which is better", make it argue for each option separately. That turns the AI from a tea-leaf reader into an impartial judge who sees the whole picture, not just whatever got shoved under its nose at the top of the text.

Bottom line: if you want sensible answers from a model, don't let it choose right away. Make it advocate for every option first, even the most ridiculous one, and then compare the results. That strips out systematic noise and gets the model to switch on its logic instead of taking a shot in the dark. A two-step check is the only way to get an LLM to stop agreeing with itself and start actually thinking.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on BiasPrompting: Forcing the Model to Explore Every Option Before It Chooses arXiv:2511.20086

Sign in to apply this study New accounts get 50 free credits.

Related techniques

Get the week’s best papers by email

One email on Sundays with the highest-rated new techniques. Unsubscribe with one click.

By subscribing you agree to our Privacy Policy. We only use your email to send the newsletter.