Evidence-Sufficiency Wrapper: how to make a model admit incomplete data instead of confidently hallucinating

One prompt, four blocks — executed sequentially in a single response:

HAVE DATA:      [list what's known from the request]
MISSING DATA:   [list what's critically absent]
SUFFICIENT?:    [render a verdict — can you answer at all]
ANSWER:         [only if sufficient — otherwise explain what's needed]

All in one request. No separate steps, no multi-message back-and-forth needed.

Task: You're launching a new product and asking Claude to assess whether you should go B2B or B2C. Data is minimal: you have an idea, no revenue, no customer discovery, no grasp of unit economics.

Prompt:

You are a business strategy consultant specializing in go-to-market decisions.

Before giving a recommendation, run a structured analysis:

HAVE DATA: list everything I've provided and everything you know about the situation

MISSING DATA: list the key data missing for a well-grounded answer

SUFFICIENT?: render a verdict — can you give a grounded recommendation with what's available? Justify it.

ANSWER: if the data is sufficient — give a recommendation with reasoning. If not — explain exactly what data needs to be gathered first, and why.

Question: I have a SaaS product for warehouse management, MVP is ready, team of 3. Go B2B or B2C?

Result: The model will show a structured breakdown: in the "MISSING DATA" block it will spell out exactly what it needs to know — deal size, ICP, ongoing conversations, competitors. In "SUFFICIENT?" it will honestly say there isn't enough data for a final verdict. In "ANSWER" — instead of inventing a confident recommendation, it will give a concrete list of what to figure out and why it changes the decision. This is a fundamentally different kind of answer than "I recommend B2B because..." with no grounding.

LLMs default to optimizing for "a helpful answer," not "an honest answer." When data is thin, the model still fills in the gaps and answers confidently. This isn't deception — it's a pattern: the next token is supposed to be a "helpful continuation," not "stop, there's not enough data."

Models are good at listing and classifying — that's a genuine strength. If you explicitly ask it to spell out what's there and what isn't, it does that well. The research found that structured formatting alone (without an explicit instruction to hold back) already cuts overconfident answers by 38% of the total effect — simply because the structure forces the model to "think it through" before answering.

Adding an explicit instruction like "don't answer if data is insufficient" adds another 62% of the effect on top of the structure. Bottom line: structure + explicit permission to abstain = the model stops faking confidence where it has none.

Control levers: - Verdict strictness — add to the SUFFICIENT? block: "If even one key parameter is unknown, treat the data as insufficient" → the model gets stricter - Required data checklist — specify in the prompt what counts as "key": "for a strategic decision, you must know market size, current revenue, and competitors" → less subjectivity - Remove the "SUFFICIENT?" block → you're back to default behavior, though the structure alone still helps a bit

You are a {role}.

Before answering, run a structured analysis:

HAVE DATA: list everything provided in the request and everything you know about the situation

MISSING DATA: list the key data missing for a well-grounded answer on {task_type}

SUFFICIENT?: render a verdict — can you give a grounded answer with what's available? Justify it in one sentence.

ANSWER: if the data is sufficient — {what_to_do}. If not — list, in priority order, what needs to be found out and why it's critical to the decision.

{task}

Placeholders: - {role} → consultant, lawyer, analyst, doctor, editor — whoever fits the task - {task_type} → a strategic decision / a legal assessment / a technical recommendation - {what_to_do} → give a recommendation / draft a plan / assess risks - {task} → the actual request

🚀 Quick Start — paste into chat:

Here's a template for the Evidence-Sufficiency Wrapper. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the role, task type, and exactly what the answer needs to deliver — because without that it can't correctly formulate the "SUFFICIENT?" block and the final instruction. It'll take the template's structure and adapt it to your context.

⚠️ Doesn't do much with complete data: If all the information needed to answer is already there, the wrapper barely changes behavior — the effect is statistically null. The technique matters specifically when data is thin.

⚠️ The model can swing into hyper-caution: On some models, the wrapper amplifies abstention so much that the model starts refusing to answer even when there's plenty of data. In the study, one model's diagnostic accuracy dropped from 75% to 17% — it started abstaining almost every time. If you notice the model asking for clarification too often, soften the instruction: replace "don't answer" with "answer, but with an explicit caveat about limitations."

⚠️ Different models, different effect sizes: The direction is consistent (more caution), but the magnitude varies a lot. Test it on your own task.

⚠️ Measuring the effect is tricky: The study found that different evaluators produce different numbers. In practice: don't expect to precisely "measure" the improvement — but the qualitative difference will be obvious.

The team ran a clever experiment: they took 1,200 clinical questions and had four top models answer each one twice — once with a standard prompt, once with the wrapper. The key trick was full pairing: same question, same model, two conditions. This removes random noise and allows for an honest comparison.

But the most interesting part is the control arms. The researchers wondered: what if the judge (another LLM scoring the answers) just sees the structural tags and marks it "safe" — rewarding form, not substance? To test this, they built three versions: a neutral scaffold (same tags, no abstention instruction), the original wrapper (tags + "abstain if data is insufficient"), and a harmful scaffold (same tags + an explicit instruction to "always give a concrete answer"). The result was elegant: identical tags produced 23%, 38%, and 80% unsafe answers respectively. The judge is clearly reading content, not just tags.

Additionally, three physicians blind-rated the answers — and found the automated judge was overly strict: it flagged 44% of answers as unsafe, while the physicians only saw a real problem in 7% of them. This doesn't discredit the technique — the direction is right — but the headline number "25-percentage-point improvement" should be read as an upper bound, not gospel truth.

1. Adaptation for iterative tasks

🔧 Add a "sufficiency threshold" → control strictness

After the SUFFICIENT? block, add: "A grounded answer requires knowing at least: {list_of_critical_parameters}. If even one is missing, the data is insufficient."

This turns the model's subjective verdict into an objective checklist.

🔧 Replace ANSWER with HYPOTHESES → research mode

When what you need isn't "give me advice" but "help me think":

Copy

ANSWER: if data is insufficient — instead of refusing, formulate 
2-3 hypotheses, each with an explicit statement of what assumption it rests on. 
Flag which hypothesis would collapse first if that assumption turns out wrong.

The model stops going silent and starts structuring the uncertainty instead — which is often more valuable than an answer.

2. Extrapolation: combo with role-based debate

When the task is complex and you want to stress-test the wrapper with another layer, you can ask two "agents" to assess the same situation:

You are a {role}. Use this format:

HAVE DATA: [what's known]
MISSING DATA: [what's absent]
SUFFICIENT?: [verdict]
ANSWER:

First answer as the Skeptic — look for reasons why the data is insufficient.
Then answer as the Pragmatist — look for what can be decided right now despite the gaps.
Finally: which of these two positions is closer to the truth, and why.

{task}

The wrapper's structure stays intact, but now the model simulates two viewpoints — and you can see exactly where the Skeptic and the Pragmatist diverge on the question of "is there enough data to act."

Paper: Judge-dependent safety gains and model-specific helpfulness costs of evidence-sufficiency prompting in clinical LLMs

Author: Koyar Afrasyab, M.D. — Kinvectum AB, Sweden

ORCID: https://orcid.org/0009-0009-3530-4606

Datasets: Real-POCQi (Feng et al.), HealthBench (OpenAI), MedRBench

Reporting standards: DECIDE-AI, CONSORT-AI/SPIRIT-AI, STARD-AI