ONE PROMPT, THREE STAGES:

1. ANALYSIS → The model considers EACH answer option separately
2. DISTRIBUTION → Assigns probabilities so they sum to 1
3. CHOICE → Final answer = the option with the highest probability

It all happens in one request. The key "sum = 1" constraint is built into the instruction.

Task: You're a marketplace consultant. A client asks: why did Amazon sales drop after they changed the listing's main photo?

Prompt:

You are a marketplace analyst. A client changed the main photo of a product listing on Amazon, after which sales fell by 40%.

Possible causes:
A) The new photo shows the product worse (color, size, details)
B) The photo doesn't match what shoppers in the category expect
C) Coincidence with a seasonal dip or an algorithm change
D) Competitors launched promotions at the same time
E) Technical problems with the listing's indexing

Analyze each option separately. Then distribute probabilities across the options so that they sum to 1. Give your final answer in the format: "Most likely cause: X".

Result: The model will walk through each option in turn: which facts support it and which argue against it. Then it outputs a distribution like: A: 0.45, B: 0.25, C: 0.15, D: 0.10, E: 0.05. The final answer points to A, but a 45% probability reflects the uncertainty more honestly than the usual "it's probably the photo" with no numbers.

The LLM weakness: Models are prone to overconfidence. When you ask "give an answer and rate your confidence", the model focuses on confirming the option it chose and ignores the alternatives. It's like a defense attorney who only sees the arguments in their client's favor.

The LLM strength: Models follow mathematical constraints well and can compare options if you explicitly ask. "Sum = 1" is a simple rule the model easily satisfies.

How the method exploits this: The "sum = 1" constraint creates a zero-sum game among the options. To give one answer a high probability, the model has to "take" it from the others. That triggers deeper analysis: the model starts looking for arguments FOR every option, not just its "favorite". The reasoning gets longer (more tokens) but more honest.

Control knobs:

• Number of options → more options = more detailed analysis, but also more tokens
• "None of the above" → add it for open-ended questions to capture the "tail" of unlikely answers
• Output format → you can ask for the distribution first and the explanation second, or the other way around

For multiple-choice questions:

{task_context}

Options:
{options_list}

Analyze each option separately, weighing the arguments for and against. Then distribute probabilities across all options so that they sum to 1. Give your final answer in the format: "Answer: X".

For open-ended questions:

{question}

First, come up with the 3-5 most likely answers. Add an "Other" option for all remaining possibilities. Analyze each option separately. Distribute probabilities so that they sum to 1. Give your final answer.

Placeholders:

• {task_context}: a description of the situation and the question
• {options_list}: options A, B, C... or ones generated by the model

⚠️ Math problems: The method works worse on tasks with a single correct answer (math, code). There it's hard to generate meaningful alternatives: the answer is either right or wrong. The researchers showed a drop in effectiveness on OlympiadBench.

⚠️ Easy tasks: On easy questions (MedQA in the study) the advantage over standard methods is minimal. The method shines on hard tasks with real uncertainty.

⚠️ Tokens: The method costs more tokens because the model generates longer reasoning. That's the price of good calibration.

A team from Tsinghua University compared three ways of verbalizing confidence: Verbalized Confidence (one answer + a percentage), Verbalized Top-k (the top 2 answers with probabilities) and Verbalized Distribution (a distribution over all options). They tested 6 models, from Qwen3-4B to GPT-4.1 and DeepSeek-V3, on 5 datasets of varying difficulty: medical questions, MMLU-Pro, multi-hop QA.

The key finding: the harder the task, the bigger the advantage of the distribution. On the easy MedQA all methods are close. On the hard MedXpertQA the difference in Brier Score (a calibration metric) is almost twofold. Interestingly, small models benefit more from the method; apparently they especially need structure to assess themselves honestly.

The researchers also trained models with RL using a reward for accuracy + calibration. Verbalized Distribution converged faster and kept its edge after training. An analysis of reasoning patterns showed that the method activates "uncertainty awareness": the model more often admits the limits of its knowledge and considers alternatives.

Context: The prompt for open-ended questions (HotpotQA): the model has to consider options and distribute probabilities.

Now please answer the following question. Give your best
3 guesses in your thinking process and then also consider
"None of the above". You should think about each of them
one by one, explaining the evidence behind each option.
Then Output your final answer followed by a probability
distribution across the 4 possibilities with your final answer
at the beginning. The probability values should sum to 1.
Your final answer should be in the format: 'The answer is
{answer}. Probability Distribution: {answer}: %, {answer}:
%, {answer}: %, None of the above: %'.

💡 Adapting it for decision-making:

The method is ideal for situations with several strategies. Instead of "which option is better?", ask "distribute the probability of success across the options".

I'm choosing between three job offers:
A) A startup: higher risk, but stock options
B) A big corporation: stability, but slow growth  
C) Freelancing: freedom, but unstable income

My priorities: {priorities}

Analyze each option separately in light of my priorities. Distribute the probability that I'll be happy with my choice two years from now so that it sums to 1.

🔧 Technique: add "confidence in the estimate"

After the distribution, ask: "How confident are you in the distribution itself? Are there factors that could change the picture dramatically?" This adds a meta level of calibration.

Paper: "Don't Miss the Forest for the Trees: In-Depth Confidence Estimation for LLMs via Reasoning over the Answer Space"

Authors: Ante Wang, Weizhi Ma, Yang Liu (Tsinghua University, Institute for AI, AIR)