TL;DR
When you ask an LLM to estimate a risk ("what's the probability this fails?") and in the same breath tell it that one kind of mistake is many times more expensive than the other, the model changes the risk number itself, not just the decision. There's one reason for it: the model generates both the probability and the action in a single response, and the information about the cost of an error leaks into the estimate, even though the facts about the task haven't changed.
Picture this: you ask for a risk estimate on a deal and get 20%. Then you add "if we approve a bad deal, we lose 10 times more than if we turn down a good one." Logically, the 20% should stay put; only the threshold at which you should say no has changed. In practice, though, the model starts quoting 5% one time and 35% the next. The number "drifts" depending on what it was told about the cost of an error, not on any new facts.
The fix is simple: ask for the probability on its own, without a single mention of the cost of an error, and apply the decision threshold ("approve/reject") as a separate step, by hand or in a separate request, using an explicit formula built from the error costs. The study also showed that if you have a second, independent opinion (a second expert or model), it's more accurate to combine it with the first one through a simple formula than to ask the model to "keep in mind that the second opinion is 80% reliable and adjust your estimate."
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1 (separate request, no mention of the cost of an error):
Give the model only the facts → get a clean risk probability (0-100%)
STEP 2 (separate request or calculated by hand):
Set an explicit decision threshold = Cost_of_false_rejection / (Cost_of_false_approval + Cost_of_false_rejection)
Compare the probability from STEP 1 with the threshold → get a decision (approve/reject)
(optional) STEP 3: a second opinion:
Get a probability from a second, independent source (another model, or a session with none of the first one's context)
Combine the two probabilities with a formula (don't ask the model to "trust it and correct itself")
Example in Action
Task: An online store owner is assessing the risk of working with a new supplier before making a large prepayment.
Prompt (Step 1: a separate message or a new chat):
Here are the facts about the supplier: in business for 1 year, three online reviews:
two positive, one about a 2-week delivery delay. Offers a 15% discount
for 100% prepayment. Business registration is in order, EIN verified.
Estimate the probability (from 0 to 100%) that the supplier fails to meet its obligations
(doesn't deliver the goods or delivers them very late).
Important: base the estimate only on these facts. Don't factor in any financial
consequences of being wrong; estimate only the probability of the event itself.
Prompt (Step 2: a separate message):
I have a probability that the supplier will fail to meet its obligations: {probability from step 1}.
Cost of errors:
- If I approve the prepayment and the supplier lets me down, I lose $15,000.
- If I say no and the supplier was reliable, I lose $1,500 in upside (discount + time).
Calculate the threshold with the formula: threshold = 1500 / (15000 + 1500).
Compare the probability with the threshold and tell me whether to approve the prepayment or decline.
Result: The first answer gives you a clean probability number, untouched by the money on the line. The second gives you the threshold calculation (about 9%) and an unambiguous "approve" or "decline" based on the comparison. If the cost of an error changes later (say, the deal size is different), the probability from step 1 stays valid; you only need to redo step 2.
Why It Works
An LLM doesn't store the probability and the decision as two separate internal variables. It generates text that depends on everything it sees in the prompt. If the facts about the task and the information about the cost of an error sit in the same request, the model "blends" them when it produces the number, even if it was asked to judge only the facts.
On the other hand, an LLM handles an isolated task pretty well: when the context says nothing about the cost of errors, it estimates the probability itself more honestly. And it can apply an explicit math formula (threshold = cost/(sum of costs)) accurately if asked to do it as a separate, clean calculation, without "weighing" the risks in its head at the same time.
The method physically cuts that link: the facts and the cost of an error never end up in the same request alongside "estimate the risk." The numbers stay reusable: you can take the same risk score and apply different thresholds to it for different situations.
Control knobs: - Threshold formula → change the error-cost figures to fit your situation (10:1, 3:1, 20:1); the threshold is recalculated mathematically and the probability stays untouched. - Second independent opinion → ask a second model (or a new chat with none of the first one's context) for its own estimate, then combine them with a likelihood formula, not with a text request like "take this into account and correct yourself." - Separate sessions → the cleaner the "cage" for the first request (not a single word about money, business risk, or consequences), the more honest the number.
Prompt Template
REQUEST 1 (risk estimate only):
Here are the facts: {facts about the situation/thing being evaluated}.
Estimate the probability (0-100%) of this event: {description of the negative event}.
Base it only on the facts above. Don't factor in financial or other
consequences of being wrong; they'll come up separately later.
---
REQUEST 2 (applying the decision, as a separate message):
Probability of the event: {number from Request 1}.
Cost of errors:
- If I approve and the event happens, I lose {cost_A}.
- If I decline and the event wouldn't have happened, I lose {cost_B}.
Calculate threshold = {cost_B} / ({cost_A} + {cost_B}).
Compare the probability with the threshold and give a final decision: approve or decline.
Fill in: {facts} is what you know about the situation; {description of the negative event} is exactly what's being estimated (failure, non-delivery, defects, etc.); {cost_A} and {cost_B} are the cost of each type of error in dollars or arbitrary units.
🚀 Quick start: paste this into the chat:
Here's the principle of separating the risk estimate from the decision. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about the facts of the situation and about the cost of the two kinds of errors separately, because the method requires that the probability and the decision never get mixed in one request.
Limitations
⚠️ It doesn't rescue a bad risk estimate on its own: the separation removes the distortion caused by the cost of an error, but if the model fundamentally can't tell "risky" from "reliable," the decisions will still be weak. In the study, at a very high error cost even the "cleaned" probability made the model reject everything, meaning its decisions were no better than rejecting everything without any analysis at all.
⚠️ It takes two separate requests, not one: this isn't a one-line prompt. The method takes discipline: don't mix the facts and the cost of an error in one message, even when you're tempted to save time.
⚠️ The threshold formula is a simplification: real decisions often hinge on more than money: reputation, legal risk, and so on. The formula from the paper is a starting point, not a universal calculator for every situation.
How They Studied It
The team took 720 code patches (one that passed and one that failed the tests for each of 360 issues from real repositories) and had four different LLMs ("reviewers": DeepSeek, Grok, Mistral, GPT/Codex) estimate the risk and make an "approve/reject" decision in a single prompt. The key trick: the exact same patches and facts were shown to the models twice, once with equal error costs and once with "a wrong approval costs 10 times more," and the researchers compared whether the probability number changed, even though the facts didn't.
The result: the probability shifted by 13.6–16.9 percentage points on average, far more than when the same request is simply repeated. Worse, the decisions the models made under the "expensive" prompt were worse than rejecting everything without analysis. And when the researchers took the same probabilities from the "cheap" prompt and applied the same expensive threshold formula to them, losses went down for all four models. That proves the problem isn't the decision rule; it's that the probability itself gets corrupted when information about the cost of an error sits right next to it.
They also tested a modular approach separately: the probability was requested without a single word about the decision policy, an independent second estimate (a "monitor") was added, and the threshold was applied in code. This lowered average losses and improved probability accuracy compared to the usual one-step review, though at a very high error cost even this clean approach didn't find a single patch worth approving.
Resources
When Policies Change Probabilities: Modular Decision-Making for LLM Code Review, by Rasvik Kudum, Max Corbett, Hitansh Paliwal, Romaisa Fatima, Thomas Jiralerspong, Sneheel Sarangi (July 2026). Code and data: github.com/rasvik/when-policies-change-probabilities.