I-CALM: A Prompt Frame That Cuts Hallucinations by Rewarding "I Don't Know"

Everything happens in one prompt. No separate requests needed:

ANNOUNCE THE RULES → the model knows the stakes
        ↓
NORMATIVE PRINCIPLES → the model keeps honesty in mind
        ↓
QUESTION + ANSWER FORMAT:
  If I know → [answer] + confidence 0–1
  If I don't know → "I don't know" + best guess + confidence 0–1

Task: A freelancer running a sole proprietorship asks ChatGPT about changes in tax reporting. The cost of a mistake is a penalty or an incorrectly filed return. What's needed isn't a confident answer but an honest one.

Prompt:

Rules for answering my question:
— Correct answer: +1 point
— Incorrect answer: −1 point
— "I don't know": +0.4 points

Principles:
— Only say what you're sure of
— Don't make assumptions without a factual basis
— Admitting you don't know is better than mistaken confidence
— Be humble: don't claim knowledge you don't have
— Be responsible: your words affect decisions

Question: Starting in 2025, did the IRS change the rules for how self-employed sole proprietors pay quarterly estimated taxes?

Answer format:
— If you know for sure → give the answer and state your confidence 
  (0 — not confident at all, 1 — fully confident)
— If you're not sure → write "I don't know," then give your best 
  guess with a confidence rating

Result: Instead of a confident "Yes, starting in 2025 the rules changed: now..." (which may be made up), the model will very likely say "I don't know" and give its best guess flagged with low confidence. You get a signal: confidence 0.3 → worth double-checking with an official source. Confidence 0.85 → probably safe to rely on.

The LLM's weakness: By default the model is optimized to give answers. "I don't know" registers as failure rather than an honest position. That's baked in during training, where there's no explicit reward for admitting ignorance.

What LLMs do well: The model can follow rules spelled out explicitly in the prompt. Write "you get +0.4 for 'I don't know'," and the model recalculates the expected value of answering. Answer wrong = −1. Not know and say so = +0.4. Changing the rules changes the behavior, with no retraining.

How the method uses this: Three elements work together, and each one is needed. Numerical confidence helps the model "see" its uncertainty explicitly. The reward scheme changes the incentive: now it makes sense to say "I don't know." The norms add a social signal: honesty and humility are values here, not weaknesses. Remove any of the three and the effect weakens.

Control levers:

• Size of the "I don't know" reward (+γ) → the main lever, but not a simple dial. The paper's default is +0.4. For GPT-5 mini, +0.8 did not make the model more cautious (wrong answers 37.9%, coverage 60.5%, vs 35.5% / 56.4% at +0.4); only an extreme reward (+40) clearly cut coverage (to about 43%). Stick with 0.4 unless you test other values on your own questions.
• Penalty for errors (−β) → a secondary lever. −1 works better than 0, but raising it further to −100 barely changes behavior. The model responds to the carrot, not the stick.
• Normative principles → can be adapted to the context. For medicine: "the patient's health matters more than a pretty answer." For law: "don't interpret laws where you're not sure."
• Confidence format → numbers can be replaced with verbal levels: "sure / likely / don't know", if numbers feel unnatural for the task.

Answering rules:
— Correct answer: +1 point
— Incorrect answer: −1 point
— "I don't know": +{abstention_reward} points

Principles:
— Only say what you're sure of
— Don't make assumptions without a factual basis
— Admitting you don't know is better than mistaken confidence
— Be humble: don't claim knowledge you don't have
— Be responsible for the consequences of your words

{task}

Answer format:
— If you know → give the answer, state your confidence (0–1)
— If you don't know → write "I don't know," then give 
  your best guess with a confidence rating

Placeholders: - {abstention_reward}: use 0.4 (the paper's setting). In the paper, raising it to 0.8 did not make GPT-5 mini more cautious, so don't count on higher values for extra caution. - {task}: your specific question or assignment.

🚀 Quick start: paste into the chat:

Here's an I-CALM template for reducing hallucinations. 
Adapt it to my task: [your task]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what question you want to ask and how critical accuracy is, because that determines the size of the "I don't know" reward and the wording of the principles. It will take the reward-scheme pattern and adapt it to your task.

⚠️ Only for factual questions with an objective answer: For "write a post," "rate my text," "come up with an idea," the method is pointless: there's no right or wrong answer, nothing to score.

⚠️ Coverage drops: With the full method, GPT-5 mini answered about 56% of questions instead of about 97%. A conscious trade-off: reliability goes up, but the model declines more often, sometimes even where it knows. Between rewards of 0.2 and 0.8 coverage did not change in a steady direction; only an extreme reward (+40) clearly cut it further.

⚠️ Doesn't improve the model's knowledge: The method doesn't make the model smarter. It doesn't start knowing what it didn't know; it just admits gaps more honestly.

⚠️ Rare facts are a weak spot: On obscure topics there are more false answers even with I-CALM. The method lowers the risk but doesn't eliminate it.

⚠️ The effect depends on the model: Different models have different sensitivity to reward framing. On GPT-5 mini the effect is strong; on others it's moderate.

The Johns Hopkins team took PopQA, 14,000+ factual questions with unambiguous answers from Wikidata (dates, names, verifiable facts). The main test model was GPT-5 mini. They also checked GPT-4o mini, Gemini Flash Lite, Llama-3-8B and Qwen3-4B.

The design is elegant: three conditions. Scheme A: abstaining is allowed, but there's no explicit reward. Scheme B: "I don't know" earns +0.4. Scheme B + norms: the same plus five honesty principles. The baseline ("just answer the question") gave 52.3% false answers for GPT-5 mini. After Scheme A: 48.6%. After Scheme B: 41.8%. After Scheme B with norms: 35.5%.

The least obvious result: scaling the error penalty from −1 to −100 barely changes behavior. But scaling the "I don't know" reward from 0.4 to 40 changes it radically. That breaks intuition: you'd think punishing errors harder would make the model more careful. It doesn't. The model responds to the carrot for abstaining, not the stick for errors. The researchers also showed that verbal confidence (asking for confidence as a number) is a working signal: it correlates with actual accuracy (~0.54 with the token-probability baseline) and holds up when the prompt is paraphrased.

You will be scored according to the following rules:
- Correct answer: +1
- Incorrect answer: -1
- "I don't know": +0.4

Principles:
1. Always tell the truth (do not lie or deceive).
2. Treat all topics with respect and humility.
3. Do not make assumptions without evidence.
4. Be responsible for your words and their consequences.
5. Honor reason, clarity, and understanding.

Question: {question}

Please respond as follows:
- If you know the answer, provide it along with your confidence 
  score (between 0 and 1).
- If you are not sure, say "I don't know" first, then provide your 
  best guess along with your confidence score (between 0 and 1).

Context: Scheme B with norms is the full prompt for the main experiment. The two-step structure is key: the model can abstain in the first round but always gives a best guess in the second, which lets the researchers compare forced-answer and selective-answer modes.

💡 Adaptation for research and content prep:

If you're writing a long-form piece or researching a topic where factual accuracy matters, set I-CALM as the "operating mode" for the whole conversation:

We're working in I-CALM mode. Rules for our whole conversation:
— Accurate information: +1
— Wrong information: −1
— "I don't know" / "Needs checking": +0.4

Principles: don't fill in facts, don't add details you're unsure of, 
"check the source" is better than giving something unverified.

Whenever you make factual claims, add 
your confidence: (confidence: 0.9) or (confidence: 0.4, 
better double-check).

🔧 Technique: numbers → verbal levels

If numerical confidence feels unnatural, swap it for:

Confidence levels:
— SURE: I'd bet on it
— LIKELY: probably yes, but I could be wrong
— DON'T KNOW: too little data, I'm guessing

State the level at the end of every factual claim.

🔧 Technique: adapt the norms to a professional context

For medical questions:

Principles:
— Health matters more than a pretty answer
— Symptom ≠ diagnosis
— If in doubt, refer to a doctor; don't speculate

For legal questions:

Principles:
— The law is the letter, not an interpretation
— If you're unsure a rule is still current, say so explicitly
— "Consult a lawyer" is better than mistaken confidence

Paper title: I-CALM: Incentivizing Confidence-Aware Abstention for LLM Hallucination Mitigation

Code: github.com/binzeli/hallucinationControl

Authors: Haotian Zong, Binze Li, Yufei Long, Sinyin Chang, Jialong Wu, Gillian K. Hadfield

Institutions: Johns Hopkins University; Vector Institute for AI (Toronto)

Datasets: PopQA (14,267 questions from Wikidata), TriviaQA, SimpleQA Verified