TL;DR
Over-searching is what happens when an LLM with search access (ChatGPT with web search, Perplexity, Deep Research) keeps hunting for information even when it doesn't help — or actively hurts. A model without search says "I don't know" to a question about the future, while the same model with search fires off a chain of 10+ queries, burns thousands of tokens, and still answers wrong or wishy-washy.
The researchers found a paradox: search improves accuracy on answerable questions but wrecks the ability to decline unanswerable ones. The model sees "who will be US president in 2075?" and instead of "nobody knows" it starts digging up forecasts, statistics, and expert opinions. The more reasoning (o1-style models), the stronger the effect. The noisier the search source (outdated Wikipedia, random web pages), the more searches the model runs. In a conversation the effect compounds — if the first 3 questions were answerable, the model will stubbornly try to answer the fourth too, even if it's about the unknowable future.
The cause is an information asymmetry: corpora describe what we know, not what we don't know. Only 13-22% of retrieved documents contain "there is no answer" signals (uncertainty, contradictions). When such signals are present, the model declines almost perfectly (90%+). But in ordinary search the model only sees "positive" documents (even misleading ones) and reads the missing answer as "I didn't search hard enough, try again."
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
The Problem at a Glance
ANSWERABLE question (capital of Georgia, the country):
Without search: 60% accuracy
With search: 72% accuracy ✅ improvement
UNANSWERABLE question (US president in 2075):
Without search: 55% abstention (correctly declined)
With search: 48% abstention ❌ degradation
TPC (tokens per correct answer):
Without search: 382 tokens
With search: 765 tokens ❌ twice as expensive
The more sophisticated the model (reasoning, deep research), the wider the gap.
Examples in Practice
Example 1: A question about the future
Situation: You ask ChatGPT with web search: "What will bitcoin be worth on January 1, 2026?"
What happens (over-searching): - The model searches for analyst forecasts - Finds 5-10 conflicting opinions - Searches historical data - Searches current trends - Generates something like "experts forecast between $80k and $200k"
The problem: 2,000+ tokens and 7 search queries spent, and the answer is useless — nobody knows the future. The model without search would have said "it's impossible to predict precisely."
Example 2: A question with a false premise
Situation: "How many eggs does a tiger lay?"
What happens: - The model searches "tiger eggs" - Finds articles about eggs (of other animals) - Searches "tiger reproduction" - Tries to synthesize an answer out of unrelated information
The right answer: "Tigers are mammals; they don't lay eggs."
Example 3: An ambiguous question
Situation: "What's the capital of Georgia?"
What happens: - The model searches "Georgia capital" - Finds both Tbilisi (the country) and Atlanta (the state) - Instead of asking for clarification, it lists both or picks one at random
The right answer: "Could you clarify — Georgia the country or Georgia the US state?"
Why It Works (and Doesn't)
The model doesn't know the limits of its own ignorance. A base model without search relies on internal confidence — if there's no pattern, it can say "I don't know." But once search is added, the logic flips: a missing answer becomes a reason to keep searching, not a reason to decline.
Corpora are asymmetric by nature. Wikipedia, the internet, and knowledge bases document what we know, not what we don't. For "who is president in 2075?" search finds forecasts, articles about the future, demographics — lots of positive documents (containing words from the question), but almost zero negative evidence (explicit statements that "this is unknown"). The model sees 10 relevant documents and thinks "time to synthesize an answer," not "time to decline."
Reasoning makes it worse. Models like o1 are trained to think longer and deeper. But if the task is fundamentally unanswerable, more reasoning ≠ a better result. The model generates a reasoning chain like "maybe by analogy with the past... and if I factor in the trend... let me find an expert opinion..." and fires off 5 more searches. TPC grows 3-5x, and abstention accuracy drops.
Multi-turn conversations snowball. If the first 3 questions were answerable, the model forms a pattern: "this session is about finding answers." The fourth (unanswerable) question meets an already-biased model — it tries harder to answer because the context hints that "there's always an answer here."
When negative evidence is present, it works great. If retrieval surfaces documents with explicit uncertainty signals ("unknown," "no data," "conflicting reports"), the model abstains in 90%+ of cases. The problem is that such documents are rare — only 13-22% in real-world corpora.
Mitigation Techniques
1. Abstention-aware prompt
Explicitly give the model permission NOT to answer.
Answer the question, using search if needed.
IMPORTANT: If the question is about the future, rests on a false premise,
or is too ambiguous, decline to answer.
Say "I can't answer this" and explain why.
Examples of when NOT to answer:
- Questions about the future: "Who will win the championship in 2030?"
- False premises: "How many legs does a snake have?"
- Ambiguity: "What's the capital of Georgia?" (the country or the state?)
Question: {your_question}
Effect according to the study: - Abstention accuracy: 50% → 60% (a 10-point improvement) - TPC drops by ~25% - Answer accuracy barely suffers (71% → 70%)
2. Few-shot with abstention examples
Show the model 2-3 examples of a correct refusal.
Here's how to handle different types of questions:
Example 1 (the future):
Q: Who will be president of France in 2040?
A: That's impossible to predict. Events and elections 15+ years out
depend on countless unpredictable factors.
Example 2 (false premise):
Q: In what year did the Earth become flat?
A: The question rests on a false premise. The Earth is shaped like
a spheroid and has never been flat.
Example 3 (ambiguity):
Q: How much does a Tesla cost?
A: The question is ambiguous. Please clarify: Tesla stock,
a Tesla car (which model?), or something else?
Now your question: {your_question}
Effect: - Abstention accuracy: 50% → 63% - The best balance of accuracy and cost
3. Self-evaluation (a two-step prompt)
First assess whether the question is answerable, then answer.
STEP 1: Assess the question
Check:
- Is it about the future/the unknown?
- Is there a false premise?
- Is the question unambiguous?
Question: {your_question}
Verdict: ANSWERABLE / UNANSWERABLE
---
STEP 2: Act
If ANSWERABLE → search for information and answer
If UNANSWERABLE → explain why you can't answer
🚀 Quick start — paste into the chat:
Here's a self-evaluation template for reducing over-searching.
Adapt it to my task: [your task].
[paste any of the templates above]
The model will ask for the specifics of your question and the kind of information involved. It'll take the answerability-check pattern and build it into its workflow.
When to Use It
✅ Use these techniques if: - You work with ChatGPT (web search), Perplexity, or Deep Research - You ask questions that may have no answer - You want to save tokens/money - You're in a multi-turn conversation where context accumulates - Your search source is noisy (the open web vs. a clean knowledge base)
⚠️ Especially critical for: - Reasoning models (o1, DeepSeek-R1, Kimi-K2) — they over-search the most - Questions about the future, forecasts, the unknown - Questions with potential ambiguity - Long sessions (5+ messages) — the snowball effect
Limitations
⚠️ The effect is moderate: The techniques improve abstention accuracy by 10-13 points but don't fully solve the problem. Models still tend to search on unanswerable questions.
⚠️ Trade-off with answer accuracy: An abstention-aware prompt sometimes lowers accuracy on answerable questions by 1-2 points — the model gets more cautious.
⚠️ Doesn't work for Deep Research mode: Deep multi-step agents (like o1 Deep Research) are so geared toward exhaustive search that prompts barely help. TPC can reach 38,000+ tokens per question.
⚠️ Depends on the corpus: If the retrieval corpus has little negative evidence ("no answer" signals), the effect is weaker. It works well with high-quality sources (fresh Wikipedia) and worse with noisy ones (stale data, the random web).
How the Research Was Done
The researchers built OverSearchQA — a benchmark of 1,188 questions, split evenly between answerable and unanswerable. Three unanswerable categories: (1) Answer Unknown — about the future and unsolved problems ("who is president in 2075?"), (2) False Premise — false assumptions ("how many eggs does a tiger lay?"), (3) Underspecified Context — ambiguity ("capital of Georgia?" — the country or the state?).
The clever design trick: for each unanswerable question they matched an answerable counterpart of similar length and semantics (embedding similarity). This removes artifacts — if the model fails, it's not because the question was oddly phrased but because the problem really lies in the question type.
They tested 10 models (from GPT-4o-mini to o1, from Llama-3.2-3B to Qwen-235B) in two modes: without search (baseline) and with search (Wikipedia, the web, noisy corpora). For each question they looked at: (1) answer accuracy on answerable questions (did it answer correctly?), (2) abstention accuracy on unanswerable ones (did it correctly decline?), (3) TPC — how many tokens were spent per correct outcome.
The key finding: when an LLM judge sorted the naturally retrieved documents into "positive" (supporting an answer, even if misleading) and "negative" (uncertainty signals), it turned out that only 13-22% were negative. But in experiments where negative documents were present, abstention accuracy jumped to 90%+. This showed that models CAN abstain — the corpora just don't give them the signals.
They also tested multi-turn conversations (1-9 messages before the final question). If the conversation history contained answerable questions, the model stubbornly tried to answer the unanswerable one too. If the history was unanswerable, abstention actually improved. The snowball effect: the model forms an expectation from context.
They measured not just accuracy but the cost of failure. They broke TPC down by outcome type: correct answer, correct abstention, wrong answer, wrong non-abstention. Abstention failure (answering an unanswerable question) turned out to be the most expensive — the model runs 5-10 searches, generates long reasoning, and still ends up wrong.
The mitigations (abstention-aware, few-shot, self-eval) delivered a 10-13 point improvement but didn't fix the problem fundamentally. Even with the prompts, models still over-search — just less. Reasoning models (o1, Qwen-Thinking) showed the worst results — the deeper the reasoning, the stronger the over-searching. Deep Research mode reached absurd levels: 38,000 TPC — 221 times more expensive than the base model, with abstention accuracy of just 30%.
Resources
Over-Searching in Search-Augmented Large Language Models — the paper describes the phenomenon, the metric (TPC), the OverSearchQA benchmark, and mitigation techniques.
Authors: a team from UC Berkeley, Stanford, Google Research, and others (names aren't given in the excerpt, but judging by the quality, the work comes from top labs).