TL;DR
Add “answers from other participants” to a prompt and the model starts drifting toward them. That's conformity (the tendency to go along with the majority). This study measures how strong the pull is and under what conditions. The key condition is how close the other answer is to what the model would have said on its own: it happily joins options that are similar in meaning and almost never joins ones that are far off.
The classic pain point: you run several agents, or show the model “what your colleagues think,” and expect an independent check. Instead, the model just picks up the party line. The reason is simple: a ready-made answer showed up in the prompt, and for a model “keep going in the same direction” is easier than pushing back. The effect grows with the number of “votes” only up to five: between five and ten votes the difference almost disappears, and for some models the shift at ten is actually smaller than at five.
Here's how it's measured: ask the model the same question a hundred times, get a distribution of answers (how often each word gets picked), then repeat with a fake “consensus” in the prompt and see how the distribution moves. The practical takeaway: you don't need a crowd to steer a model, just a few “votes” for an option that's close in meaning.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Ask a “fill in the blank” question 100 times → word frequencies (the model's favorite words)
STEP 2: Come up with words the model never produced itself: “near-synonym” / “somewhat similar” / “completely different”
STEP 3: Ask a multiple-choice question (4 options) 100 times WITHOUT other answers → baseline share
STEP 4: Same thing with a block “Participant 1: …, Participant n: …” (n = 1, 2, 5, 10), everyone votes for the less frequent option
STEP 5: Compare the shares → shift in percentage points
Every step is a separate request, each in a fresh chat.
Example in Practice
The study's method is a lab setup, so instead of the experiment itself we take the conclusion: how to check whether your “independent” review is just picking up someone else's line.
Task: You're choosing a name for a prompting course aimed at small business owners. Three draft names were proposed earlier by “fellow agents.” You want a judge model to rate them independently, with no majority pressure. For that you need a blind mode: first a rating without anyone else's opinion, then a comparison.
Prompt (step 1, blind rating):
You are rating names for an online course, “Prompting for Small Business Owners.”
Names:
A. “Prompts Without the Fluff”
B. “AI for Your Business in 7 Days”
C. “Talk to AI Like It's Your Employee”
Pick the single best name for a launch on Instagram and a Substack newsletter. Give only the letter and one reason.
Prompt (step 2, same question with “opinions”):
[same text]
Here are 3 answers from other raters:
Rater 1: B
Rater 2: B
Rater 3: B
Pick the single best name. Give only the letter and one reason.
Result: In step 1 the model gives its own pick and a reason. In step 2 you'll see whether the pick changed. If it did, “other people's opinions” act as pressure, and you can't use this setup for an independent check. Repeat each step 5–10 times: a single answer tells you nothing, you need a share.
Why It Works
The model's weakness. It doesn't “have opinions” in the human sense. It picks words by probability, and text in the prompt moves those probabilities. A list of several identical answers is a strong signal that “this is how people answer.”
The model's strength. It's good at telling meanings apart. That's why other answers that are far off in meaning barely work: a word that clearly doesn't fit the sentence won't get picked, no matter how many “participants” name it. What works is something close: a synonym of what the model would have written anyway, or one of its own frequent picks.
How to use it. You decide whether to show the model other answers. For an independent rating, don't. For a robustness check, do the opposite: slip in some “pressure” and see whether the answer holds.
Control levers: - Number of votes. The effect grows up to five, then barely changes. No point adding ten “agents” when three to five will do. - Closeness of the option. If the option is a “near-synonym” of the model's own answer, it's easiest to sway. For robustness, test exactly those close options, not obviously dumb ones. - Presentation format. In the study, other answers came as a list, “Participant 1: ….” Any similar format (reviews, votes, colleague comments) works as the same signal.
Prompt Template
A template for checking whether the model is picking up the majority line.
Question: {question}
Options:
A: {option_1}
B: {option_2}
C: {option_3}
D: {option_4}
[Include this block only in the second run]
Here are {number} answers from other participants to this question:
Participant 1: {pressure_option}
...
Participant {number}: {pressure_option}
Answer with a single letter.
What to plug in:
- {question} and the options: your choice task (a name, a priority, a support-ticket category, a verdict).
- {pressure_option}: the one the model picks less often. Ideally something close in meaning to its frequent pick.
- {number}: 1, 2, 5. Going above five isn't worth much.
Compare the share of answers with and without pressure, 20–100 runs per mode.
Limitations
⚠️ Narrow task: Everything was measured on ten fill-in-the-blank sentences, ESL-quiz style. The study says nothing about reasoning, risk analysis, or writing.
⚠️ Artificial pressure: The “participants” are just lines in the prompt, and they're all unanimous. In real multi-agent systems the votes differ, and the picture may change.
⚠️ No tested defenses: The authors didn't test how to reduce conformity (a “think independently” instruction, shuffling the order, hiding other answers). Those tricks remain guesses.
⚠️ Averages hide the spread: Models differ a lot in how far they shift: one shifts almost three times as much as another. Test your own model.
⚠️ Distribution estimates are approximate: Distributions were estimated from a hundred runs, not computed exactly.
How They Studied It
The idea was simple: port Solomon Asch's classic experiment (a person answering under pressure from a unanimous group) to LLMs. But lines on cards mean nothing to a model, so the task was swapped for filling a word into a sentence, which is exactly what a model was trained to do.
First, each of six top models (OpenAI, Anthropic, Google) got ten sentences a hundred times each, and the researchers recorded which words it picked. Words that showed up in at least 5% of runs were labeled favorites. Then a separate model came up with words the tested model had never produced, at three levels of closeness: near-synonym, somewhat similar, completely different. Next, the models got a choice of four words (including two “fillers”) with no hints, and with hints like “Participant 1: …” where every participant named the less frequent option. Group sizes were 1, 2, 5, and 10. Each condition got a hundred runs in a clean chat at temperature 1.
Result: all six models conform. The strongest pull is toward another one of their own favorite words: the shift grows from 22 points with one “vote” to 49 with ten. Toward a near-synonym the shift is also large (30 points at ten), toward “somewhat similar” it's 24, toward “completely different” it's zero. The biggest surprise: between five and ten votes the growth practically stops, and for some models the shift at ten is even smaller. The spread across models is huge: from 33 points for one Anthropic model to 73 for one of the Google models.
Practical takeaway: to quietly drag a model far from its answer, all you need is a chain of small, plausible steps, each backed by a couple of votes. One big leap won't move it. That matters both for attackers and for anyone building multi-agent pipelines.
Adaptations and Extrapolations
🔧 Technique: strip “colleagues' opinions” out of the judge's prompt → get an independent rating
In “generator → judge” pipelines, the judge is often handed previous ratings. Instead, give the judge only the material itself and the criteria, and compare ratings in a separate step.
Rate the text against these criteria: {criteria}. Text: {text}.
You don't see any other ratings; judge based on the text alone.
Extrapolation: checking how robust an answer is
Take a high-stakes decision (say, a verdict on a contract or picking a pricing plan) and run it with a block saying “3 other lawyers chose option B.” If the answer changes, don't rely on a single model run and add a human review.
The authors didn't test these tricks; they're ideas based on their results.
Resources
A Quantitative Approach to Asch Conformity in Frontier Large Language Models. Senthil Rajasekaran (Université libre de Bruxelles), Stylianos Loukas Vasileiou (New Mexico State University). Builds on the classics: Asch (1956), Deutsch & Gerard (1955), plus work on LLM conformity: Zhu et al. (2024), Weng et al. (2025), Choi et al. (2025), Baltaji et al. (2024), Bito et al. (2026), Bajaj & Tiganj (2026).