Contradiction Gate: a separate check-request that catches contradictions missed while solving the task

STEP 1 (request 1): Solve the task based on the data → a number OR "I can't answer"
STEP 2 (request 2, separate, same model): Check this same data for completeness
and internal consistency → label: "all good" / "missing data" / "contradiction found"
STEP 3 (logic on top): If STEP 2 = "contradiction found" →
      replace the STEP 1 answer with a refusal, regardless of what it said

Steps 1 and 2 are two separate requests (in the study, two separate API calls with the same context). In an ordinary chat, that's two messages or two separate chats with the same source data.

Task: You're a freelance project manager. A client sends you a brief for a website build. The brief contains a contradiction: at the start, the client writes "budget is $800, a simple one-page landing page," and at the end — "we also need a product catalog section with 200 items and a user account area." You ask ChatGPT to work out a quote, and the model will most likely just calculate the quote based on one version of the requirements, without noticing the conflict.

Prompt (request 1 — solving):

You are an experienced web development project manager. Here's a client brief:

"Budget is $800, need a simple one-page landing page.
Also need a product catalog section with 200 items and a user account area."

Estimate the approximate cost and timeline for this project based on this brief.
If there isn't enough data to calculate — write "cannot calculate."

Prompt (request 2 — verification, as a separate message or in a new chat):

Check this client brief for internal consistency:

"Budget is $800, need a simple one-page landing page.
Also need a product catalog section with 200 items and a user account area."

Are there any facts in this brief that contradict each other
(e.g., different scope of work, different budgets, different deadlines)?
Answer with one of three options:
1) "everything is consistent"
2) "missing data" — specify what's missing
3) "contradiction found" — name specifically which two facts conflict

Result: In the first request, the model will most likely silently calculate a quote — say, for a "simple landing page" — and never mention that a 200-item catalog and a user account area obviously don't fit that budget. In the second request, the model will most likely call out the conflict directly: "a simple one-page landing page" doesn't square with "a 200-item catalog and a user account area" at an $800 budget. Once you see that flag, you throw out the quote from the first request and go back to the client for clarification instead of handing over a confidently wrong number.

When the model solves a task, all its attention goes into the computation: it takes the facts in order and calculates, without cross-checking them against each other — much like a person absorbed in doing long division who doesn't notice the word problem states two different ages for the same character. The model simply doesn't spend "attention budget" cross-referencing facts, because the task is framed as "calculate this," not "check this."

But the model's strength is that it's genuinely good at spotting inconsistencies when explicitly asked to compare facts against each other rather than compute something. The instruction "check for contradictions" is a different task with a different focus of attention, and in that mode the model catches the conflict 75-83% of the time — almost regardless of how strong the model is at math.

The method exploits this gap: instead of improving the calculation itself, it adds a second look at the same data with a different task and uses that as a filter before handing the answer to the user.

Control levers: - Verification labels (all good / missing / contradiction) → expand these for your use case, e.g., add ambiguous wording - Wording of the verification question → the more specific it is (e.g., "compare the budget figures across different parts of the text"), the higher the chance of catching a conflict - What to do when the gate triggers → in the study this is a hard refusal, but you could swap that for "show both calculation scenarios" instead of a flat refusal

REQUEST 1 (solving):
You are a {expert role}. Here's the input data:

"{data/brief/task description}"

{specific instruction — calculate/solve/draw a conclusion}.
If there isn't enough data to answer — write "cannot answer."

---

REQUEST 2 (verification, as a separate message):
Check this same input data for internal consistency:

"{same data/brief/task description}"

Are there any facts in it that contradict each other?
Answer with one of three options:
1) "everything is consistent"
2) "missing data" — specify exactly what's missing
3) "contradiction found" — name specifically which two facts conflict

---
If the answer to REQUEST 2 is "contradiction found," do not use the answer
from REQUEST 1. Instead, state that the data is contradictory
and specify exactly what needs to be clarified.

What to fill in: {expert role} — who the model is impersonating (accountant, lawyer, project manager), {data/brief/task description} — the source text with the facts, {specific instruction} — exactly what needs to be calculated or resolved.

🚀 Quick Start — paste into chat:

Here's a template for the "Contradiction Gate" technique — a two-step check
of data for contradictions before trusting the model's answer.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste template above]

The LLM will ask what kind of data you typically feed in and what type of contradictions tend to show up in it (numeric, categorical, temporal) — because the data type determines exactly how to phrase the verification question in the second request.

⚠️ Extra cost: the method requires a second request to the model for every task — that's extra time and tokens, even if the amount is small.

⚠️ False positives: the verification request sometimes finds a "contradiction" where there isn't one, and throws out a correct answer. In the study, this cost up to 5 percentage points of accuracy on clean data.

⚠️ Not tested on every type of conflict: the method reliably proved itself on contradictions in categorical facts (status, category, yes/no). For numeric contradictions (two different values for the same amount), the researchers note the model tends to read these as a "clarification" or "update" rather than an outright conflict — the gate may perform worse here.

⚠️ Same model for both solving and verifying: if the model has a "blind spot," it might fail to catch the same type of error in both modes. Using different models for solving versus verifying hasn't been tested, but it's potentially more reliable.

The researchers took 91 tax cases from the well-known legal benchmark SARA (tax calculation under real U.S. tax code) and created two kinds of "corrupted" versions: some had a required figure removed (e.g., income), and others had a contradicting categorical statement inserted (e.g., "filing jointly" next to "filing separately"). They tested six recent models — GPT-5 mini, GPT-5.2, Claude Sonnet 4.6, Qwen3.7-Plus, Kimi K2.5, Gemini 2.5 Flash — running each three times for reliability.

Each model was asked either to solve the task (give a number or refuse) or, separately, to check the input data for contradictions. It turned out that when a fact was missing, models almost always honestly refused to answer. But with a contradiction present, the four most accurate models refused only 0-15% of the time and, in 63-76% of cases, simply produced an answer as if there were no conflict at all. Yet those same models, when asked to "check the data," caught the conflict 75-83% of the time.

An interesting detail: models with "chain of thought" (i.e., ones that "think" before answering) weren't any better at finding contradictions than models without that mode — meaning it's not about depth of reasoning, but about where the model directs its attention depending on whether the task is framed as "calculate this" versus "check this." That's exactly where Contradiction Gate comes from: a simple way to make the model take a second look at the data before you trust its first answer.

Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems, Albert Sadowski, Jarosław A. Chudziak, Warsaw University of Technology. Dataset and code: publicly available on Zenodo (doi.org/10.5281/zenodo.22329077). Uses the SARA / LegalBench benchmark (Holzenberger et al., 2020; Guha et al., 2023).