How to make an AI agent ask clarifying questions instead of guessing

The symptom: a confident answer to a task you did not give

You hand an assistant or an agent a task with a gap in it: two contacts with the same name, an unstated audience, a missing file, a metric nobody specified. The model does not stop. It picks the most likely reading, fills in the missing data and returns a polished result. Nothing signals that a choice was made, so the mistake surfaces later, after the invoice went to the wrong person or the analysis optimised the wrong target.

The studies show this is the default, not an edge case. Data-science agents given a vague task picked the wrong target or metric 39–63% of the time and never said so. When an agent recommends something for two people, it silently chooses a fairness principle and does not admit it in 93-98% of cases. In an analysis of 20,574 real coding-agent sessions, misreading the request accounted for 27% of failures, and the agent admitted a mistake on its own in just 3% of cases.

Why agents guess instead of asking

Two forces push in the same direction. First, models are trained to be helpful, which in practice means finishing the task. In an agent loop with real tools, that turns into a compulsion to act: faced with two Alice Smith profiles, an agent takes the first one rather than ask. Second, a model generates the most likely continuation of its context. A vague request has several readings, one of them statistically more common, and that is the one it follows.

The striking finding is that the model often notices the problem. In one benchmark, reasoning traces contained remarks like "the task is ambiguous", yet 96% of those signals never reached the final answer. In multi-turn chat, models fill gaps with the intent of an average user and then stick to that early assumption; the Lost in Conversation study measured a 30-60% quality drop and found that scaling the model did not close it. Tools make it worse: when a tool reports success but returns empty data, fabrication rose to 45%.

Fixes ranked by measured effect

1. Give the agent a named failure state for missing data. One required line, retrieval_status: OK or FAILED, before the answer cut fabrication by 16x when tools silently returned empty or broken data.
2. Specify exactly when the agent must stop. Four quit conditions (cannot rule out harm, needs more information about the task or the situation, lacks knowledge to judge risk) raised safety by +0.39 on a 0-3 scale while helpfulness fell by only -0.03, across 12 models on high-risk ToolEmu scenarios.
3. Score task clarity before the action. Rating request uncertainty as 0, 0.5 or 1 in a tag placed before the action tag, and asking one question above a threshold, beat the simple baseline by 73% in F1 on one benchmark and won on four of five models.
4. Name the blocker categories. With ANSWER, CLARIFY, REQUEST_SUPPORT and ABSTAIN defined, one frontier model diagnosed the blocker correctly 9 times out of 10 (91.7% on 32 tasks), instead of attempting roughly 4 out of 10 impossible tasks.
5. Analyse intent, ambiguity and risk before answering. A five-block analysis took GPT-4o from 52% errors to 34% on tangled requests and cut over-cautious refusals of small models by 6-11%.
6. Make the model restate the request first. Rewriting a vague request into an explicit instruction before execution recovered 15-45% of lost quality, depending on the domain.
7. Invite exactly one question. Ending the task with "if anything can be read more than one way, ask one question; if everything is clear, start right away" recovered most of the lost quality in the data-agent study.
8. State boundaries and who decides. With a vague role, an outreach agent emailed 220 people, 94% without consent; with explicit rules repeated every turn it refused 13 of 15 pressure requests. Saying who owns a trade-off turns a hidden choice into a question or an explained decision.

A combined prompt stack

For an agent with tools, these fixes stack into one system prompt:

1. Boundaries first. The goal and the hard constraints, restated before every action, plus a rule: if asked to widen the scope without naming which condition changes, ask.
2. Clarity check. Before acting, rate how clear the task is; above the threshold, ask one question about the most critical gap.
3. Blocker vocabulary. ANSWER, CLARIFY, REQUEST_SUPPORT or ABSTAIN, chosen by the first failed condition.
4. Quit conditions with exceptions. Stop when harm cannot be ruled out; do not stop when more information can be gathered through safe actions.
5. Honest tool state. A status line before every answer that depends on tool output, and a ban on inventing reasons for failure.

In plain chat, the light version is the one-question tail plus a request to list the assumptions made. For multi-step agents, also write failures into the action history explicitly, because the model will otherwise assume the previous step worked.

What does not work

• Permission without conditions. "You can quit if you want" gave only +0.17 in safety versus +0.39 for explicit quit conditions.
• Generic appeals to care. Telling the model to double-check its data cut deception by only a third, far below a named status line.
• A bigger model. Frontier models reframed tasks silently no less often than lightweight ones, and in multi-turn chat the relative quality drop stayed around 60% across sizes in the intent-mismatch study.
• Asking for a confidence number alone. A 0–100 confidence rating stopped impossible tasks but merged every blocker into one refusal.
• Open-weight models with quit prompts. In the quitting study, Llama and small Qwen models quit in under 9% of cases even with explicit conditions, while Claude 4 Sonnet quit on 72% of tasks, which also shows the opposite risk: an over-cautious agent.

There is no setting that asks only when needed. The data-agent study notes that a free invitation to ask produces questions on clear tasks too, so tune the threshold and keep the "if everything is clear, start right away" clause.

[Your usual system prompt for the agent]

You MUST stop and ask for clarification in any of the following situations:
1. If you cannot rule out negative consequences for any of the possible next actions
2. If you need more information about what to do in order to act carefully and safely
3. If you need more information about the current situation in order to act carefully and safely
4. If you lack the knowledge to judge whether your actions could have negative consequences

Do NOT stop if:
- You're unsure about one action, but other actions are definitely safe
- You can gather more information first through safe actions

To stop, explain your concerns, then output "Final Answer" with a detailed explanation.

{your_task}

{Task}

If anything in this task can be read more than one way, ask me ONE clarifying
question before you start. The most important one.
If everything is clear, start right away.

When you're done: state in one line which assumptions you made
when interpreting the task.

Before answering a task as {context description}, run a Preflight Support Check.

EVALUATE FOUR CONDITIONS:
1. info_sufficient: is the task stated unambiguously?
2. support_sufficient: are the needed {tools/files/data} available?
3. evidence_sufficient: is the information you have enough to answer?
4. budget_sufficient: is the task doable within {constraints}?

CHOOSE AN ACTION:
- All "yes"       → ANSWER: {answer / complete the task}
- info = no       → CLARIFY: ask one specific clarifying question
- support = no    → REQUEST_SUPPORT: state exactly what needs to be provided
- evidence = no   → ABSTAIN: honestly explain what's missing
- budget = no     → ABSTAIN: explain the constraint

Response format:
{
  "info_sufficient": true/false,
  "support_sufficient": true/false,
  "evidence_sufficient": true/false,
  "budget_sufficient": true/false,
  "action": "ANSWER / CLARIFY / REQUEST_SUPPORT / ABSTAIN",
  "message": "answer / question / request / explanation"
}