TL;DR
Stating both probabilities explicitly is a simple trick: instead of "60% chance of success" or "40% chance of failure," you put both numbers in the prompt at once: "60% chance of success, 40% chance of not succeeding." The model gets the probability in a single, balanced form and stops latching onto whichever outcome you happened to mention first. In the study, this made the advice almost completely independent of the wording.
Here's the pain point: the same probability, described from different sides, produces different advice. Say "80% it'll work" and the model confidently recommends going ahead. Say "20% it won't work" — identical odds — and the answer gets more cautious, with a higher risk rating. In a third of cases the advice category changes. And it's almost never "yes" flipping to "no"; it's "yes" or "no" sliding into "not sure". Outright reversals happen less than 1% of the time, which is exactly why the problem is easy to miss: the answer looks perfectly reasonable. The cause is that the model reacts to the words in the text ("chance of not...", "risk") rather than recomputing the number in its head.
The method is simple: one step, one prompt. Add the second number. A second technique — asking the model to work out the complementary probabilities itself and rely on them — helps less: advice changes drop from 34% only to 23%. Stating both numbers explicitly brings it down to 7%, which is the level of ordinary noise between repeated runs.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
BEFORE: "40% chance of success. Should we launch?"
or "60% chance of failure. Should we launch?"
→ the answer depends on which version you wrote
AFTER: STEP 1: Take the outcome probability → write down both sides
STEP 2: Put both numbers in the prompt in the same format
"40% chance of success / 60% chance of not succeeding"
STEP 3: Ask your decision question
(all in one prompt)
Example
Task: The founder of a D2C cosmetics brand wants to launch a new product line on Amazon. The inventory order costs $30,000. By the founder's estimate, there's a 40% chance of turning a profit within 3 months. They ask an AI assistant whether to launch or not.
Prompt (the wrong way):
We want to place a $30,000 inventory order for a new product line and launch it on Amazon.
The probability that we won't break even within 3 months is 60%. Should we launch?
Framed this way, the model is more likely to retreat into a cautious "not sure" or a "no," and to rate the risk higher. Ask it as "40% chance we break even" and the advice may come out softer.
Prompt (the right way):
We want to place a $30,000 inventory order for a new product line and launch it on Amazon.
Outcome probabilities over 3 months:
- 40% — we turn a profit (break even and make money)
- 60% — we don't turn a profit (partially recoup or end up in the red)
Total: 100%. This is one and the same estimate, just described from two sides.
If we don't break even, we lose up to $14,000 of our free cash.
If we do, we make about $22,000 in profit.
Give a recommendation: launch / don't launch / need more data.
Explain which numbers you're basing it on.
Result: The model gives one of three recommendations plus an explanation grounded in both numbers and the upside and downside amounts. The answer shouldn't change if you rewrite the inputs "from the other side." For extra confidence, run the prompt twice with the lines in swapped order: the advice should match.
Why It Works
The weakness. The model reads words, not numbers. "40% chance of not getting it" sounds like a warning; "60% chance of getting it" sounds like a promise. To a human they're the same thing; to the model they're different signals. The model mirrors human behavior: the gain-loss framing effect has been documented in people since 1981. It hits risk assessment hardest, along with whether the model will commit to a decisive recommendation.
The strength. When both numbers sit side by side in the text, the model sees that it's a single distribution. It doesn't have to subtract from 100% on its own — and that's precisely the step that breaks when the number is framed as a loss.
Control levers: - Both numbers in the same format (both percentages, same order) → removes the skew. Don't write "60%" alongside "one in two." - The line "This is one and the same estimate" → locks in that the two numbers aren't two scenarios. This is my addition; the authors didn't test it. - Explicit upside and downside amounts → shift focus from emotionally loaded words to numbers. - An answer format of "launch / don't launch / need more data" → lets you see right away whether the frame pushed the model into "not sure."
Prompt Template
{description of the situation and the decision}
Outcome probabilities:
- {p}% — {positive outcome}
- {100-p}% — {negative outcome, the complement of the first}
Total: 100%. This is one and the same estimate, described from two sides.
Consequences:
- If {positive outcome}: {what happens}
- If {negative outcome}: {what happens}
Give a recommendation: {answer options}. Explain which numbers you're basing it on.
What to fill in:
- {p} and {100-p} — always both probabilities, adding up to 100%.
- Phrase outcomes as mirror pairs: "we break even / we don't break even," "treatment works / treatment doesn't work."
- {answer options} — give the model a legitimate third option ("need more data"). That's exactly where it escapes to when the frame shifts, and this way you'll notice.
Option 2 (weaker): the "find the complementary probability" instruction. It works worse, but it's useful when you don't control the input (for example, when a client sends you their text). The exact wording isn't given in the accessible part of the paper; the version below is reconstructed from the method description:
Before making a recommendation, list every probability in the description and find
its complement to 100%. Base your reasoning on the structure of the probabilities, not on
the words they're phrased in. Then give your recommendation.
Limitations
⚠️ Fights noise, not error: the method makes advice stable, not correct. If the model gives bad advice, it will give equally bad advice under any frame.
⚠️ The weak substitute, "find the complement": it cuts advice changes by roughly a third but doesn't eliminate them. If you can rewrite the input yourself, give both numbers.
⚠️ Residual noise: about 7% of matches still "drift" between runs. That's normal model variability, and the method doesn't remove it.
⚠️ Small-scale test of the fix: the technique itself was tested on three models and 32 scenarios. The framing-dependence problem was found across seven models, but the fix was validated on a smaller set.
⚠️ Binary outcomes only: "success / failure." The effect wasn't tested for complex distributions (multiple outcomes, ranges).
⚠️ Experimental conditions: temperature 0, strict JSON output, and synthetic scenarios generated by GPT-5 and hand-filtered. Results in a live chat may differ, although the effect held up on conversational paraphrases.
⚠️ Weaker models suffer more: in small models (e.g., GPT-4o-mini) the advice category changes in more than half of cases. In models at the level of Claude Sonnet 4.6 or Gemini 3 Flash the effect is smaller, but not zero.
How They Tested It
The idea was simple: take a decision with a fixed probability and change only which side it's described from. The authors built 240 scenarios across four domains: health, finance, education and career, and consumer rights. Each scenario was written at probabilities of 20, 40, 60, and 80% and in three frames: neutral, "chance of success," and "chance of failure." That came to 2,880 prompts. Two people checked that "success" and "failure" described exactly the same decision without hidden emotional wording.
Seven models (GPT-4o-mini, GPT-4.1-mini, GPT-5, Claude Sonnet 4.6, Gemini 2.5 Pro, Gemini 3 Flash, Qwen3-VL) answered in JSON: a recommendation (yes/no/not sure) plus ratings of risk, "nudging toward action," and confidence on a 1–5 scale. Result: the advice category changed in 32.4% of cases. 98.1% of those changes involved "not sure," while a direct "yes ↔ no" reversal happened in just 0.63%. The most common transition was "not sure" under the "chance of success" frame → "no" under the "chance of failure" frame (1,301 cases). Risk ratings were higher on average under the "chance of failure" wording. The effect showed up in all seven models: 63% changes for GPT-4o-mini, 22% for Qwen3-VL. The lower the probability of success, the stronger the skew: at 20% the risk shift was 0.535, at 80% it was 0.109.
Three people independently rated 140 answer pairs blind and confirmed that risk and directiveness really are perceived differently. Then the authors checked whether this was an artifact of templates: they rewrote the scenarios in conversational language (3 models, 32 scenarios, 3 repeats each). The effect stayed at the same level (36.8% → 34.2%). The key contrast: repeated runs of the same wording diverge in 7% of cases, while across frames it's 34%. So this isn't ordinary noise.
Finally, they tested two fixes. Stating both probabilities explicitly cut advice changes from 34.2% to 6.7% and nearly erased the risk shift (from 0.363 to −0.010). The "find the complement yourself" instruction only got it to 22.8%. The practical takeaway: it's easier to fix the input than to ask the model to correct itself.
Adaptations and Extrapolations
These are my own ideas inspired by the paper; the authors didn't test them.
🔧 Technique: double-frame run → self-check of the advice
Ask the same question twice, in mirrored frames, in two separate chats: "40% chance of success" and "60% chance of not succeeding." If the advice category differs, don't trust either answer. Give the model both numbers at once and ask again.
🔧 Technique: input normalization in the system prompt → in the instructions for an assistant that receives probabilities from users (support, sales, an internal analyst), add:
If the user states a probability from only one side, rewrite it from both sides before answering ("X% yes / (100−X)% no") and base your answer on both.This is a weaker form of the method (essentially the "find the complement" instruction), so for critical decisions it's better to normalize the input data before it reaches the model.
Resources
- Paper: Same Probabilities, Different Advice: Gain-Loss Framing Effects in High-Stakes LLM Decision Support
- Authors: Yuyao Zhang, Yakun Cui, Han Zhu, Jinsong Lu, Yike Guo, Sirui Han — Hong Kong University of Science and Technology, Beihang University
- Benchmark: FRAMEBENCH-HPS (2,880 prompts, 240 scenarios)
- Underlying theory: Tversky & Kahneman, 1981 (framing effect); related work DeFrame (Lim et al., 2026)