TL;DR
MABPD is a method where three expert roles examine a text from different angles (tone bias, evidence base, framing of facts), then their verdicts get weighted by a formula: role weight × confidence × presence of an actual quote from the text. If an expert shouts "bias here!" but can't quote the exact spot in the text — their vote gets zeroed out, no matter how confident they claim to be.
A single LLM playing bias-detector tends to hand down a confident "gut-feel" verdict without actually checking — and after self-critique, it just doubles down on its first opinion instead of reconsidering it. Simply running a few "agents" in parallel barely helps: without a structured debate, recall (the ability to actually catch biased texts) drops by half — the model gets too cautious and misses half the real cases.
The core of the method isn't the number of roles — it's the burden-of-proof rule: a bias accusation without a quote = zero weight. But a neutral verdict without explicit quotes still gets partial weight (0.6) — because "I looked carefully and found no skew" is a signal too. The final judge (verifier) can only downgrade a "biased → neutral" verdict, never upgrade it — so the system never overreaches.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1 (in a single prompt, roles are independent of each other):
Role "Tone" (weight 0.5), Role "Evidence" (weight 0.25), Role "Framing" (weight 0.25)
each produces → verdict (biased/neutral/uncertain) + up to 5 quotes + confidence 0-1 + a counterargument against itself
STEP 2 (weight calculation):
effective_weight = role_weight × confidence × evidence_coefficient
coefficient = 0 if "biased" with no quote
coefficient = 0.6 if "neutral" with no quote
coefficient = 1 if a quote exists and is verified in the text
STEP 3 (voting):
the verdict with the highest sum of effective weights wins
"biased" only counts if at least one quote is actually verified
STEP 4 (final check):
The judge may downgrade "biased" → "neutral" (if the evidence is weak)
The judge CANNOT upgrade "neutral" → "biased"
Example Use Case
Scenario: You run a small brand, and an influencer just posted a scathing review of your product. You're not sure whether it's fair criticism or a biased hit job (maybe the reviewer has a conflict of interest with a competitor). You need to figure this out before responding publicly.
Prompt:
You are playing the role of three independent experts checking a text for bias.
Each expert has their own focus area and vote weight.
Expert 1 — "Tone" (weight 0.5): looks for judgmental, emotionally loaded words,
name-calling instead of factual description.
Expert 2 — "Evidence" (weight 0.25): checks whether claims are backed up
by concrete facts, sources, examples — or are just bare assertions.
Expert 3 — "Framing" (weight 0.25): looks at what the author left out, whose viewpoint
is missing, whether there's a manipulative structure (e.g., only negatives with no context).
Each expert must produce:
— Verdict: "biased" / "neutral" / "uncertain"
— Up to 5 direct quotes from the text as evidence (exact phrases, not paraphrases)
— Confidence from 0 to 1
— A counterargument against itself (what could disprove its own verdict)
EVIDENCE WEIGHT RULE (must be applied):
— If the verdict is "biased" and there isn't a single direct quote — evidence weight = 0
(the expert's vote doesn't count, regardless of confidence)
— If the verdict is "neutral" with no explicit bias-related quotes — evidence weight = 0.6
(the absence of bias signals is itself a signal)
— If a quote exists — evidence weight = 1
Calculate the final verdict as a weighted majority:
role_weight × confidence × evidence_weight.
To declare the text "biased," at least one expert must provide an actual quote.
At the end, run a final check: if the overall verdict is "biased" but the evidence is
weak (few quotes, low confidence) — downgrade it to "neutral."
Never upgrade "neutral" to "biased" at this step.
Text to analyze:
{blogger review text}
Result: The model will show three separate expert assessments with quotes and confidence levels, then a weight calculation and a final verdict with an explanation of why (e.g., "Expert 'Tone' called the text biased but provided no quotes — its vote didn't count; Expert 'Evidence' found 2 unverified claims — that weight was counted"). At the end you get a single final answer: biased or neutral, grounded in specific quotes rather than a vague impression.
Why This Works
A single model, when simply asked "is this text biased?", tends to render a verdict based on overall gut feeling — fast, but unreliable: it might sense "something's off" and call the text biased with no real grounding in specific wording. Worse: if you ask it to double-check itself, it more often confirms its first opinion than reconsiders it.
On the other hand, an LLM is genuinely good at finding and quoting specific passages when explicitly asked to, and it can hold multiple "viewpoints" within a single reasoning pass if given clear roles.
The method leverages this: it forces every "role" to mandatorily back up its verdict with a quote, then mathematically zeroes out unsupported votes. That way, confidence can no longer stand in for evidence.
Control knobs: - Role weights (0.5/0.25/0.25) → adjust if your task cares more about facts or framing than tone. - The "neutral without evidence" coefficient (0.6) → raise it if you want to trust a "clean" verdict more; lower it if you want to demand evidence even for neutral conclusions. - The threshold for the final verdict (0.5 in the original) → raise it to 0.7 if you need stricter filtering for clear-cut cases. - The judge's asymmetry (can only downgrade) → this is the key lever against false positives; if missing real cases worries you more than avoiding false accusations, you can make it symmetric.
Prompt Template
You are playing the role of {number} independent experts analyzing {text_type}
for {what_to_check — bias/manipulation/unsupported conclusions}.
Expert 1 — "{role_1}" (weight 0.5): {what_it_checks}
Expert 2 — "{role_2}" (weight 0.25): {what_it_checks}
Expert 3 — "{role_3}" (weight 0.25): {what_it_checks}
Each expert produces:
— Verdict: "{option_1}" / "{option_2}" / "uncertain"
— Up to 5 direct quotes from the text as evidence
— Confidence from 0 to 1
— A counterargument against itself
EVIDENCE WEIGHT RULE:
— Verdict "{option_1}" without a quote → evidence weight = 0
— Verdict "{option_2}" without explicit quotes → evidence weight = 0.6
— A quote exists → evidence weight = 1
Final verdict = sum (role_weight × confidence × evidence_weight) for each option,
the winner is whichever has the max. To declare "{option_1}," at least one expert must provide a quote.
Final check: if the overall verdict is "{option_1}" with weak evidence —
downgrade to "{option_2}." Never upgrade in the opposite direction.
Text to analyze:
{text}
What to fill in: {text_type} — article, review, letter, business proposal; {what_to_check} — bias, manipulation, unreliability; roles — pick 2-3 angles suited to your task (e.g., for resume screening: "facts," "exaggerations," "omissions").
🚀 Quick start — paste into the chat:
Here's a template for a multi-agent bias check with a burden-of-proof rule.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which three angles matter for your type of text and which pair of outcomes (biased/neutral or some other pair) you're checking for — because the weights and evidence rule are tuned specifically to that pair of verdicts.
Limitations
⚠️ Short texts: the method requires quotes as evidence. Tweets, short comments, or headlines just don't have enough material for 5 verified quotes — the roles will be spinning their wheels.
⚠️ Overkill on easy cases: the full cycle (3 roles + weight calculation + final check) is overkill for a text that's obviously neutral or obviously toxic. The original research has a "fast path" for this — in chat, you'll need to add it separately.
⚠️ Mostly tested on news text: transferring this to other genres (ads, personal posts, business correspondence) hasn't been validated in the research — the burden-of-proof logic may need the role wording adapted.
⚠️ The judge's asymmetry reduces sensitivity: the system is deliberately tuned to avoid overreach (it won't inflate a "biased" verdict). If missing manipulation worries you more than false accusations do, you'll need to remove the asymmetry.
How It Was Studied
The team used the BABE dataset — 4,121 news sentences labeled for bias by expert annotators — and compared MABPD against classic approaches: fine-tuned BERT, RoBERTa, and the current SOTA (MAGPIE, a model trained on 59 auxiliary tasks). All roles in MABPD ran on the same model (LLaMA 3.3 70B) — deliberately, to isolate the effect of the architecture itself rather than any particular model's "smarts."
The key experiment was an ablation: removing the debate module and keeping only the voting rules → the score dropped 9-10 F1 points, mostly because the model got too cautious and missed half the genuinely biased texts. Removing all roles except one → dropped another 9-19 points. This proves the gain isn't just "more agents beat one" — it's specifically the burden-of-proof rule and the debate between roles.
They also tested across different models — swapping two supporting roles for weaker models (Mixtral, LLaMA 8B) — and the result barely changed (a 0.1-point difference). This shows the gain comes from the weighted-debate structure itself, not from having a "smart" model inside it. They additionally tested transferring the method to a different dataset (SemEval HyperPartisan, articles rather than sentences) — the result nearly matched supervised SOTA without a single labeled training example, confirming the method isn't overfit to one specific dataset.