Uncertainty Decomposition: Two Signals Instead of One — Separating "The Task Is Unclear" from "I Don't Know How to Do It"

Everything happens in a single prompt, one step after another:

STEP 1: Rate the task uncertainty (ut)
        → 0 (all clear) / 0.5 (some gaps) / 1 (critical details missing)

STEP 2: Routing
        → if ut ≥ 0.5: ask a clarifying question (stop)
        → if ut < 0.5: move on to the action

STEP 3: Perform the action + rate your confidence in it (ct)
        → 0-1, how confident you are that you're moving toward the goal

STEP 4: Write both signals to the history
        → the next step will take them into account

The key point: ut is rated BEFORE the model generates an action — so the task's ambiguity doesn't get a chance to "infect" the solution itself.

Task: You're the founder of a small EdTech startup, raising money. You ask Claude to help with your pitch for Y Combinator.

Prompt:

You are an assistant who works step by step and always checks that the task is clear before acting.

Before every response, you must use this format:

<thinking>
[reasoning about the task and what needs to be done]
</thinking>

<task_uncertainty>
[a single value: 0, 0.5 or 1]
0 — the task is completely clear, there is only one correct interpretation
0.5 — there are gaps, the user most likely has a preference they haven't stated
1 — critical details are missing, too many equally valid interpretations
</task_uncertainty>

<uncertainty_explanation>
[what exactly is unclear — or why the task is considered clear]
</uncertainty_explanation>

<action>
[if task_uncertainty ≥ 0.5 → ask a clarifying question, do not perform the task]
[if task_uncertainty < 0.5 → perform the task]
</action>

<confidence>
[0.0–1.0, how confident you are that the action moves toward the goal]
</confidence>

<confidence_explanation>
[why this level of confidence — what helps, what worries you]
</confidence_explanation>

Task: Help me write a pitch for my Y Combinator application.

Result:

The model won't start writing the pitch right away. The <task_uncertainty> block will show 1 or 0.5. In <uncertainty_explanation> you'll get a list of concrete gaps: what's the product? what stage is the startup at? which batch are you applying to? what do the YC partners look for? In <action> — a clarifying question about the most critical one. Only after you answer does the model move on to real help — with <confidence> in the 0.7–0.9 range and an explanation of what it took into account.

If you give it a specific request ("write a market-problem slide for a B2B EdTech product, the audience is YC partners with a technical background, we have 3 minutes for the pitch") — the model will set ut = 0 and get straight to work.

The LLM's weakness. A model is a next-token generator. When a task is fuzzy, it doesn't freeze up and ask. It fills in the gaps on its own — picks the most likely interpretation and keeps going. In a short exchange you won't notice; in long multi-step tasks, one early assumption drags a whole chain of wrong decisions behind it.

The LLM's strength. If you explicitly ask a model to rate how clear an instruction is, it does a good job. Models are good at spotting missing details and phrasing questions. The problem isn't ability — it's that without an explicit instruction, the model skips this step and jumps straight to generating an answer.

How the method exploits this. The XML structure turns "rate the clarity" from a nice-to-have into a mandatory step before acting. The model can't step over <task_uncertainty> and <uncertainty_explanation> — they come earlier in the text than <action>. The order of the tags sets the order of thinking.

Control knobs:

• Trigger threshold (≥ 0.5) → lower it to ≥ 0.3 if you want more clarifying questions, raise it to ≥ 0.8 if your tasks are usually clear and you don't want extra questions
• The 0/0.5/1 scale → can be extended to 0/0.25/0.5/0.75/1 for finer granularity
• History → drop the <confidence> and <uncertainty> blocks from the history if you're saving tokens; put them back when tasks are long and multi-step
• Tag names → rename them in your own language; they're not syntax, just hints to the model about the structure

You are an assistant who works step by step and checks that the task is clear before acting.

Respond to every request strictly in the following format:

<thinking>
[reasoning: what the task is, what's clear, what isn't, how I plan to proceed]
</thinking>

<task_uncertainty>
[a single value: 0, 0.5 or 1]
0 — the task is completely clear, one correct interpretation
0.5 — there are gaps, the user most likely has a preference
1 — critical details are missing, too many interpretations
</task_uncertainty>

<uncertainty_explanation>
[what specifically is unclear, or why the task is considered clear]
</uncertainty_explanation>

<action>
[if task_uncertainty ≥ 0.5:
  — ask one clarifying question about the most critical gap
  — do not perform the main task]
[if task_uncertainty < 0.5:
  — perform the task]
</action>

<confidence>
[a number from 0.0 to 1.0: how confident you are that the action moves toward the goal]
</confidence>

<confidence_explanation>
[what makes you confident, what worries you, what's missing]
</confidence_explanation>

Task: {task}

Placeholders: - {task} — your request; paste it as is, even if it's vague — the method will find the gaps itself

🚀 Quick start — paste into the chat:

Here is the Uncertainty Decomposition template. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what kind of tasks you usually work on and how specific they tend to be — because the optimal trigger threshold depends on that (0.5 by default, but lower is better for creative tasks).

⚠️ Simple, unambiguous tasks: The method adds extra steps with no payoff. If you ask "translate this sentence into Spanish," a clarity check is pointless — you're just burning tokens.

⚠️ Token overhead: Six tags at every step make the conversation noticeably longer. For short tasks, simplify — keep only <task_uncertainty> and <action>.

⚠️ Systematic LLM overconfidence: Models tend to underrate ut — putting 0 where it's really 0.5. This is a known problem for all prompt-based confidence-estimation methods. If you notice the model rarely asks questions, lower the threshold.

⚠️ Biggest payoff is in long chains: The method's main value shows up in multi-step tasks (an agent taking 10+ actions in a row). For one question, one answer, the gain is smaller.

A team from ITMO University (St. Petersburg) ran into a real problem: existing methods for estimating LLM-agent confidence either require access to internal token probabilities (not available through APIs), run the model several times (expensive and slow), or need training on labeled data. That left only one viable niche — prompt-based methods.

The idea was simple: what if you split one uncertainty signal into two? To test it, the researchers built two new benchmarks — WebShop-Clarification (an agent shops for products online) and ALFWorld-Clarification (an agent does household tasks in a simulation). In both, exactly half the tasks are deliberately written vaguely: the agent has to either spot the gap and ask, or guess wrong. That made it possible to measure not just "how confident the agent is" but "how accurately the agent recognizes that a task is unclear."

They tested five different models — GPT, DeepSeek, Qwen, GLM — and compared three approaches: simply appending confidence to the end of the prompt (ReAct+UE), confidence with memory (UAM), and the proposed two-signal method. The gain turned out to be substantial — on one of the benchmarks the new method beat the simple baseline by 73% in F1, which is surprising for a prompt-only method with no fine-tuning. Tellingly, the method won on four of the five models, so the effect isn't a fluke and isn't "tuned" to a single LLM.

An interesting detail: the researchers deliberately excluded from the comparison every method that requires code or fine-tuning — not out of laziness, but because in real-world conditions those are unavailable. That makes the results more honest: they compare things you can actually use.

Here is the exact agent output format from the paper (Appendix A.3, formula 11):

<think> rt </think>

<u_request> ut ∈ [0, 1] </u_request>

<u_request_explanation> explanation of what's unclear </u_request_explanation>

<action> at </action>

<confidence> ct ∈ [0, 1] </confidence>

<explanation> explanation of the confidence </explanation>

With routing from Algorithm 1:

if ut ≥ θ (default θ = 0.5):
    at* ← request_clarification
else:
    at* ← at

Both variables — ut and ct — and both explanations are written to the agent's history so later steps can refer back to them.

Context: This is the exact prompt format the authors tested on five LLMs. The ut scale is deliberately three-point (0/0.5/1) rather than continuous — that way models calibrate better and get stuck on in-between values less often.

1. Adaptation: A short version for chat (no agent)

If you don't need memory across steps and just want protection against a "confidently wrong answer" to one tricky question:

💡 Adaptation for one-off tasks: Drop the history and the multi-step setup, keep only the core check.

Before answering the task:

<uncertainty>
0 — the task is clear / 0.5 — there are gaps / 1 — critically few details
</uncertainty>

<if_unclear>
[if ≥ 0.5: ask one question, stop]
[if < 0.5: answer]
</if_unclear>

Task: {task}

2. Technique: A persona instead of a faceless agent → sharper critique

🔧 Swap "agent" for a specific role → the model plays it more precisely

Instead of a generic "you are an assistant," add a role:

You are a seasoned product manager from Google who has seen hundreds of vague specs.
Before taking on a task, you always check: do we actually understand this task the same way?

This changes the behavior: the model starts asking sharper, more professional clarifying questions, because "a PM from Google" knows exactly what people usually forget to specify.

3. Extrapolation: Combo with Chain-of-Thought

The "rate the clarity first, then act" principle can be built into any CoT prompt as step zero:

Before thinking about the solution:

Step 0: Are there any ambiguities in the task? 
[yes/no + what exactly, if yes] → [if yes: ask for clarification]

Step 1: [regular CoT...]

This works even without the XML structure — just numbered steps with a step zero for the clarity check.

Paper: Matsnev, G. (2026). Uncertainty Decomposition for Clarification Seeking in LLM Agents. Preprint.

Code: github.com/PE51K/udcs-in-llm-agents

Author: Gregory Matsnev, AI Talent Hub, ITMO University, St. Petersburg

Related methods from the paper: ReAct (Yao et al.), UAM — Uncertainty-Aware Memory (Zhang et al.), Self-Consistency (Wang et al.), Semantic Entropy