TL;DR
The method separates the moment of verification from the model's history of actions: instead of asking the LLM to evaluate the result of its own query or conclusion inside the same conversation, you show it only the "clean" facts, with no trace of its own reasoning. The model judges as if it's seeing the information for the first time, rather than defending its past decision.
It turns out the model isn't honest with itself. If the LLM wrote a search query itself and the results turned out to be irrelevant (even when that's spelled out right in the page's snippet), it still opens the page, simply because the query was its own. But if the very same results are presented as someone else's text, with no mention that the model came up with the query, it instantly notices the irrelevance and refuses to go further. The authors call this inertia bias: the model doesn't want to admit that its own action was a mistake.
The method fights this through context isolation at two critical points: (1) when the agent decides whether to open a page it found, it's shown only the task facts, without the history of its own search query; (2) when the agent prepares the final answer, its chain of reasoning is broken into numbered steps and each transition (A→B, B→C) is checked separately, in isolation from the overall context.
Method Blueprint
BLOCK 1 — Page check (search):
STEP 1: The agent runs a query → gets a list of snippets
STEP 2: An isolated filter looks at the snippets WITHOUT the query history →
decision: "open the page" / "rephrase the query"
BLOCK 2 — Final answer check (a separate call after the model has decided it's ready to answer):
STEP 1: Reorganization — scattered reasoning steps are turned into an explicit
numbered chain: A → B → C → ...
STEP 2: Step-by-step check — each transition (A→B, B→C) is verified
separately, as a standalone claim, without relying on the fact
that the previous step has "already been approved"
STEP 3: If a weak transition is found → a specific note and a revision.
If all transitions hold up → the answer is released
Both blocks can be run as separate requests to the model. In the original this is an automatic step inside an agent system, but the principle carries over easily to a regular chat, done by hand.
Example in Practice
Task: You asked ChatGPT to gather facts for due diligence before investing in a startup, for example to verify the claim "company X's revenue grew 300% in a year according to public sources". The model found sources and drew a conclusion. Now you want to know whether it's passing off wishful thinking as fact, simply because it has already "bought into" its own conclusion.
Prompt (for a separate, fresh chat; that's the context isolation):
Here is a claim and the sources it relies on,
with no explanation of who found them or how:
Claim: "Company X's revenue grew 300% in a year"
Source 1: [quote/data]
Source 2: [quote/data]
You are an independent auditor who did not take part in selecting these
sources and did not see how the claim was formulated.
1. Break the conclusion into a chain of logical steps: A → B → C.
2. Check each transition separately: does B follow from A factually
and logically? Don't treat a step as valid just because the previous
step looks plausible.
3. Point out specifically where the logic breaks down (if it does).
4. Give a verdict: "All steps confirmed" or "Problems found: [list]".
Result: The model will build an explicit chain of reasoning from the sources to the conclusion, check each transition separately, and point to the specific step where the logic is weak (for example, "source 2 talks about growth over a quarter, not a year"), instead of a generic "yes, it all looks convincing".
Why It Works
When an LLM sees in the context that it itself formulated a query or conclusion, it tends to defend that decision rather than evaluate it objectively, like a person who doesn't want to admit their own idea was bad. This is an internal bias, not caving to the user's opinion (which sets it apart from sycophancy), and not simply clinging to an old hypothesis (confirmation bias): the trigger is specifically authorship of the action.
On the other hand, an LLM is good at judging objectively when it doesn't see the "this is my action" link. In the experiments, the model almost always correctly discarded irrelevant results when the same data was presented as outside information, with no mention that it had come up with the query.
The method exploits that gap: it removes the "I decided this" trace from the context, leaving only the task facts. The model judges as an outsider, not as the author.
Control levers: - Granularity of the check (the whole answer at once vs step by step) → for simple tasks an overall check is enough; for important decisions, break it into steps - Degree of isolation: you can use a "new chat" as maximum isolation (the model doesn't see at all that it's its own text), or simply remove phrases like "you decided that..." from the prompt - Number of checkpoints: the original has two nodes (search and the final answer); in a chat you can apply just one, wherever the risk of error is higher
Prompt Template
Here is the final conclusion for the task "{task}":
{conclusion_without_mention_of_how_it_was_reached}
You are an independent expert who did not take part in creating this
conclusion and did not see the process of reaching it.
1. Break the conclusion into a chain of logical steps: A → B → C → ...
2. Check each transition separately: does the next step follow from
the previous one factually and logically? Don't treat a step as valid
automatically just because the previous step looked right.
3. If you find a weak spot, state specifically what the problem is
and propose a fix.
4. Give a final verdict: "All steps confirmed" or "Problems
found: [list]".
Fill in {task} with the essence of the decision being checked (a plan, a calculation, a fact), and {conclusion_without_mention_of_how_it_was_reached} with the answer itself, scrubbed of phrases like "I decided" or "based on my search" (that's the isolation).
🚀 Quick start: paste this into the chat:
Here's a template for an isolated check of a conclusion. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask exactly which conclusion needs checking and at what point in the reasoning it appeared, because an honest check requires separating the conclusion itself from the history of how it was made.
Limitations
⚠️ Manual work: in the original, automation does the job (the agent scrubs the context itself). In a chat, the user has to manually extract a "clean" conclusion with no traces of the model's own reasoning. If you just copy the whole conversation, the effect disappears.
⚠️ Scope: the method was tested on tasks with a verifiable factual answer (dates, numbers, statistics). For subjective assessments (creative work, writing style) the paper offers no clear confirmation of the effect.
⚠️ It doesn't eliminate errors entirely, only reduces the risk: even with context isolation the model can make the same mistake again. The method reduces the bias; it doesn't guarantee a correct answer.
How They Tested It
The team first built a diagnostic test, IBIS: they had the qwen3-max model generate over a thousand factual questions with verifiable answers, weeded out the weak ones via evaluation by three different LLMs (Gemini, DeepSeek, Claude) on verifiability and answer uniqueness, and kept 712 high-quality examples. Then real search queries were run through Google, and human annotators (with inter-annotator agreement of 0.68 Fleiss' kappa, a decent level of agreement) labeled whether, given these results, you should search again, open a page, or whether the answer is already in hand.
The key experiment: the same search results were given to the model in two forms, as its own action (Agentic Mode) and as outside text (Observer Mode). The gap in accuracy between these modes is a direct measurement of inertia bias, and it turned out to be substantial: the model was much worse at admitting its mistake when the query was "its own".
After that, NIS-Agent was tested on web-agent benchmarks (GAIA, WebWalkerQA, BrowseComp, BrowseComp-zh) across several models (GPT-4o, GPT-4.1, Claude 3.7, DeepSeek). The method performed on par with the best systems but used a third fewer tokens, because the isolated filter immediately discards useless pages. Separately, they fine-tuned a small 8-billion-parameter model (via SFT and GRPO reinforcement learning) so that it would have less tendency toward inertia bias from the start, and that small model came close to GPT-4o in quality. That part requires code and training, but the fact itself, that "baked-in" resistance to the bias works just like manual context isolation, confirms that the mechanism is real and not an artifact of the prompt.