STEP 1: PLAN → "What I'll check and how"
    ↓
STEP 2: EXECUTION → Step-by-step analysis with reasoning
    ↓
STEP 3: VERDICT → "Yes, there's a hallucination" / "No"
    ↓
STEP 4: CORRECTION → Rewrite the problem spots (if needed)

💡 All 4 steps run sequentially in ONE conversation

Task: You're a marketer and ChatGPT gave you a competitor profile for an analytics report. You need to check whether the model made up any details.

Prompt:

You are checking a text for factual errors and hallucinations.

TEXT TO CHECK:
"Etsy was founded in 2003 by Rob Kalin. The company was the first in the US to launch a marketplace for handmade goods, in 2010, and today ranks second in sales after Amazon."

STEP 1: Draft a verification plan. Which facts need to be verified? What could the sources of error be?

STEP 2: Carry out the plan step by step. Check each fact and show your reasoning.

STEP 3: Draw a conclusion: does the text contain hallucinations? Yes or No.

STEP 4: If yes, rewrite the text with the errors fixed.

Result: The model will produce a structured analysis: first the plan (check the founding date, the founder's name, the marketplace claim, the market position), then a step-by-step check of each fact with reasoning, then a verdict naming the specific errors, and finally a corrected version of the text.

The LLM weakness: asked directly "is there a mistake?", the model gives a superficial answer. It doesn't decompose the task; it just generates the first thing that comes to mind. It's like asking an expert in passing: they'll say something, but they won't think it through.

The LLM strength: models are great at following step-by-step instructions. Tell one "first draft a plan, then execute it" and it really does structure its work. Each step becomes an "anchor" for the next.

How the method exploits this: HalluClean adds a metacognitive layer: the model first thinks about HOW to check, and only then checks. Plus the "task router", a short phrase like "You are checking an answer to a question", helps the model understand the context. The researchers showed that adding the router alone improves F1 on summarization from 25% to 60%.

Control levers:

• Plan detail: for simple checks "what to check" is enough; for complex ones, add "what sources of error are possible"
• Reasoning depth: you can ask it to "show the reasoning briefly" to save tokens
• Task type in the router: "an answer to a question", "a document summary", "a reply in a dialogue"; adjust it to the context

You are checking {content_type} for factual errors and hallucinations.

{CONTENT_TYPE} TO CHECK:
{text_to_check}

STEP 1: Draft a verification plan. Which facts need to be verified? What could the sources of error be?

STEP 2: Carry out the plan step by step. Check each fact and show your reasoning.

STEP 3: Draw a conclusion: does the text contain hallucinations? Yes or No.

STEP 4: If yes, rewrite the text with the errors fixed.

Placeholders:

• {content_type}: what you're checking: "an answer to a question", "a document summary", "a reply in a dialogue", "a product description"
• {text_to_check}: the text that needs checking

Router Variants by Task Type

⚠️ Depends on the base model: the method amplifies a model's abilities but doesn't create new ones. If GPT-3.5 doesn't know a fact, it can't check it. For fact-checking in specialized fields (medicine, law), combine it with web search.

⚠️ Three types of errors remain: (1) language misunderstanding: the model misreads the question, (2) missing knowledge: the model doesn't know the fact it needs, (3) reasoning errors: it understood correctly but drew the wrong conclusion.

⚠️ More tokens: the four-step process generates more text than a direct question. For bulk checking, factor in the cost.

A team from the Harbin Institute of Technology tested HalluClean on five task types: question answering, dialogue, summarization, math problems and self-contradictions. They used four existing labeled benchmarks, about 2,000 examples in total, balanced between "hallucination / no hallucination".

They compared it with the direct question "is there a hallucination?" and with other prompting methods: Step-by-Step, Plan-and-Solve, SelfCheckGPT, ChatProtect. They tested on five models: GPT-3.5, GPT-4o-mini, Llama-3-70B, DeepSeek-V3, DeepSeek-R1.

Main insight: the direct question gives an F1 of about 33% on QA, while HalluClean gives 68% on the same GPT-3.5. Interestingly, each component contributes something. The task router alone lifts summarization F1 from 25% to 60%. Adding structured reasoning brings it to 66%. The effect is additive, not a single silver bullet.

They also tested it on domain-specific tasks (medicine, finance), where the method works even better: F1 rose from 9% to 92% on CovidQA. That makes sense: in specialized fields models make things up even more often, so structured checking catches more.

Context: The four-step prompt for the Question Answering task (verbatim from the paper):

Step-1: You are provided with a question and its corresponding answer.
Your task is to determine whether the answer contains hallucinated content.
Let's understand the task and devise a plan to solve the task.
[Task Input]

Step-2: Let's carry out the plan and solve the task step by step.
Show the reasoning process.
[Task Input; Result from Step-1]

Step-3: Please conclude whether the answer contains hallucinated content
with Yes or No.
[Task Input; Result from Step-2]

Step-4: Given a question, its corresponding hallucinated answer, and an
analysis explaining why the answer contains hallucinated content.
Your task is to answer the question without introducing any hallucinations.
[Task Input; Result from Step-2]

Task routers (verbatim):

💡 Adaptation: checking your own writing before you send it

The method works not only for checking LLM answers but also for self-checking, when you wrote a text yourself and want to catch the errors:

You are a fact-checking editor. Check my text for factual errors.

MY TEXT:
{your_text}

STEP 1: Which facts in the text can be checked? List the claims.

STEP 2: Check each claim. Show your reasoning.

STEP 3: Which claims are definitely correct, which are doubtful, which are wrong?

STEP 4: Suggest fixes for the doubtful and wrong ones.

🔧 Technique: add web search for fact-checking

The researchers showed that HalluClean + retrieval gives an F1 of 80% instead of 68%. In a chat with Claude or ChatGPT:

You are checking an answer to a question for hallucinations.
When checking facts, search the web.

QUESTION: {question}
ANSWER: {answer}

STEP 1: Which facts need to be checked via search?
STEP 2: Find the information and compare it with the answer.
STEP 3: Verdict: is there a hallucination? Yes/No.
STEP 4: If yes, fix it.

🔧 Technique: shorten it for simple checks

If you're checking something simple (a date, a name, a single number), you can drop the detailed plan:

Check this fact: {claim}

Show your reasoning and give a verdict: true or not?
If not, correct it.

Paper: "HalluClean: A Unified Framework to Combat Hallucinations in LLMs"
Authors: Yaxin Zhao, Yu Zhang, Harbin Institute of Technology, Harbin, China
Code: https://github.com/tingmuor/HalluClean
Benchmarks: HaluEval, UMWP, ChatProtect, HaluBench