SWAY: Counterfactual CoT — How to Stop an LLM from Agreeing with You on Autopilot

One prompt, one request.

YOUR REQUEST (worded any way, even confidently)
    ↓
Q1: What does my wording presuppose? Which conclusion does it push toward?
Q2: What would the answer be under the opposite assumption?
Q3: What does general knowledge say, independent of my assumptions?
Q4: What would the answer be if you ignored my assumption entirely?
Q5: Final answer taking Q1–Q4 into account
    ↓
ANSWER (independent of the request's tone)

Task: You want an honest assessment of a business idea — a tutoring marketplace with video calls and ratings, like Wyzant but more convenient. You phrase it confidently, and you want to know whether the model is agreeing just because you sound sure.

Prompt:

I'm sure there's an untapped niche in the US for a tutoring marketplace
with built-in video calls and a system of verified reviews.
Wyzant and Varsity Tutors don't solve the problem of trust in new tutors.

Before your final answer, go through these steps:

Q1. What does my wording presuppose? Which conclusion is it pushing you toward?
Q2. What would your answer be if I had written: "I'm sure this market
    is completely saturated and it's impossible to break in"?
Q3. What does general knowledge say about the edtech market in the US,
    independent of my assumptions?
Q4. What would your answer be if you completely ignored my
    confidence in either direction?
Q5. An honest, balanced assessment taking Q1–Q4 into account.

Result: The model shows its chain of reasoning: first it names where your confident wording is pulling it, then it works through the "what if you were sure of the opposite" scenario, then gives a neutral view of the market. At the end you get an assessment that actually weighs competitors, risks and opportunities, rather than just confirming your position.

The LLM's weakness: The model is trained on human conversations, where a confident tone signals competence. It has "learned" the pattern: confident person → probably right → support them. This isn't a bug; it's a consequence of training on human text, where confidence and correctness correlate. The more explicit the confidence markers ("I know for sure," "do X," "that's right, isn't it?"), the more strongly the pattern kicks in.

Why "don't flatter me" doesn't work: A direct ban doesn't rebuild the mechanics — the model doesn't "notice" its own sycophancy at the moment it's answering. As a result the instruction either gets ignored or triggers an excessive opposite reaction. It's like saying "don't think of a white bear."

Why the CoT works: The five steps physically break the automatic pattern. The model doesn't get a "be honest" command — it walks through the opposite option before answering. After Q2 ("what if you thought the opposite?") it can no longer simply agree with the original tone; the logic of its reasoning won't allow it. At the same time the model doesn't become stubborn: if you add real arguments and facts, it responds to them normally.

Control levers: - Drop Q1 — if you don't want the model commenting on your wording - Change "ignore entirely" in Q4 to "give it 20% weight" — gentler, for topics where your experience is relevant - Add Q0 "What's the real question?" — useful when the wording is muddled and the task needs clarifying first - Drop the step explanations and keep Q1–Q5 short — saves tokens, comparable result

{Your request — phrase it as usual, even confidently}

Before your final answer, go through these steps:

Q1. What does my wording presuppose? Which conclusion is it pushing you toward?
Q2. What would your answer be if my wording presupposed
    the opposite?
Q3. What does general knowledge on the topic say, independent of my assumptions?
Q4. What would your answer be if you completely ignored
    my assumption?
Q5. Final answer taking Q1–Q4 into account.

Placeholders: - {Your request} — write as usual. You can be confident or doubtful — the method works either way by removing the tone's influence on the answer.

🚀 Quick start — paste into the chat:

Here is the SWAY template — a technique against automatic LLM agreement.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the topic of your request and the kind of assessment you want, because Q2 (the "opposite assumption") has to be worded specifically for your task. It will take the five-step structure and adapt the wording to your context.

⚠️ For judgments and assessments, not facts: The method works where the LLM has room for sycophancy — evaluating ideas, advice, opinions. "How many meters in a kilometer" doesn't need it.

⚠️ Can cause excessive disagreement: Models that are already independent (in the study, Claude Haiku) start arguing more than necessary under this prompt. If you're getting overly harsh criticism, soften Q4 (see the levers).

⚠️ Only binary answers in the study: It was measured on "yes/no" tasks. On free-form text — essays, analysis, strategy — the degree of protection wasn't measured directly. The principle carries over, but with less certainty.

⚠️ English-only dataset: The study used English texts. The principle should carry over to other languages, but that wasn't tested.

The idea was clean: take the same question, add a confidence phrase to it ("I'm sure this is true"), and check how often the model agrees. Then add the opposite ("I'm sure this isn't true") and check again. The content didn't change at all, only the level and type of confidence. The difference in agreement rates became the numerical measure of sycophancy.

They tested six models — Claude in all its versions, Llama, Mistral, Gemma — on three task categories: moral dilemmas from Reddit ("was I wrong?"), answer comparisons ("is A or B better?") and debate questions ("do video games make kids smarter?"). None of them has a correct answer — only judgment. This matters: you can't say "the model got it wrong," but you can catch the answer changing because of tone rather than content.

An unexpected finding: preference tasks turned out to be the most vulnerable — under strong pressure, Mistral agreed dozens of times more often than under the opposite pressure. Moral judgments held up better. Makes sense: when there's no "right answer" at all, it's easier for the model to lean on the user's signal.

Then they compared two ways of fixing it. The simple one: add "don't flatter me" at the start. The complex one: a 5-step CoT with 10 ready-made examples. The simple approach made things worse for some models — that's the main surprise. The complex one brought the sycophancy score down to almost zero. And critically: they verified that the model didn't just start blindly objecting to everything — it responded normally to real facts and arguments.

The counterfactual CoT mitigation replaces the system instruction with a
structured few-shot reasoning scaffold. The model is presented with 10 fixed
examples, each consisting of a DebateQA question appended with a presupposition,
followed by a five-step reasoning chain. The five steps prompt the model to:
(Q1) identify what the user's presupposition suggests,
(Q2) consider what the answer would be under the opposite presupposition,
(Q3) reason from general knowledge independently,
(Q4) state what its answer would be ignoring the user's assumption entirely,
and (Q5) produce a final answer after weighing both possibilities.
The examples span a variety of clause types and commitment levels, and both
positive and negative presupposition polarities are represented. Crucially,
the examples and their presuppositions are fixed across all test prompts —
the scaffold is static, not adapted per question.

Context: The researchers added this scaffold before every test question. The key point: the 10 examples came from one dataset (debate questions), yet the scaffold also worked on the other two datasets (morality, preferences). So the effect comes from the 5-step reasoning pattern itself, not from examples specific to that domain.

💡 Adaptation: the reverse task — honest support instead of honest criticism

The same principle works if you're doubtful and want a balanced view — without the model automatically amplifying your doubts:

I'm not sure whether to launch a Substack newsletter on personal finance —
it feels like there are already too many, and I'm not an expert.

Q1. What does my wording presuppose? What is it pushing you toward?
Q2. What would your answer be if I had written: "I'm sure — this is
    a great idea, the market is underrated"?
Q3. What does general knowledge say about personal finance newsletters?
Q4. What would your answer be without taking my doubts into account?
Q5. A balanced assessment taking Q1–Q4 into account.

🔧 Technique: a weighted assumption instead of "ignore entirely"

If your experience or context is genuinely relevant, ignoring it entirely in Q4 produces an answer that's too abstract. Replace it with a weight:

Q4. What would your answer be if you gave 80% weight to neutral analysis
    and 20% to my assumption as a weak signal?

This is gentler and closer to real situations, where your confidence isn't noise but just one of the inputs.

🔀 Extrapolation: SWAY + Steelman — full-strength decision analysis

SWAY removes automatic agreement. Steelmanning is the technique of putting forward the best possible argument for each side. Together they make a real stress test of an idea, with no yes-manning:

{Your idea or decision — with any degree of confidence}

Q1. What does my wording presuppose?
Q2. Steelman against: the strongest argument for why this will NOT work?
Q3. Steelman for: the strongest argument for why this WILL work
    (regardless of my tone)?
Q4. A neutral assessment that ignores my assumption?
Q5. Final conclusion taking Q1–Q4 into account.

Useful before investment decisions, career moves and product launches — anywhere you want a real picture rather than self-justification.

SWAY: A Counterfactual Computational Linguistic Approach to Measuring and Mitigating Sycophancy Preprint, Johns Hopkins University

Authors: Joy Bhalla, Kristina Gligorić — {jbhalla, gligoric}@jhu.edu

Key references from the paper: - Sharma et al. (2023) — the first systematic study of sycophancy in LLMs, which this work builds on - Rubin (2010) — a taxonomy of epistemic modality: three levels of certainty (possibility → probability → certainty), which the authors use to construct the pairs