TL;DR
This study compares four ways to have an LLM verify scientific claims: no search, a single search using the claim text, an agent that writes its own queries (up to 4), and an "oracle" that's handed the right papers up front. The agent turned out to be the best of the realistic options. It turns the claim into a short query of key terms on its own, and it finds the right paper three times as often as a brute-force search.
The real pain point is elsewhere: a model without search cites real but completely irrelevant papers. The PubMed IDs exist, but the paper is about eyeglasses for kids while the claim is about cytokines. Of 327 existing citations, only 3 were relevant. So a "the link exists" check guarantees nothing. With a search-enabled agent, about 67% of citations are relevant. In that case the model doesn't hunt for confirmation in its memory — it takes papers from what it actually found.
The method is simple: let the agent reformulate the query, explain the search rules, require it to cite only what it found, and then check the citations with a separate request: "does this paper study exactly these entities and the relationship in the claim?" The bottleneck isn't reasoning, it's search: when the model was given the right papers, accuracy rose from ~60% to 79–86%.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: The model gets the claim + search rules → a short query (2–6 key terms)
STEP 2: Search → show only new results → the model decides: search more or answer
(limit of 4 searches; after the limit, a forced answer)
STEP 3: Verdict → JSON: label (SUPPORT / CONTRADICT / NEI), cited_pmids, rationale
(it may only cite what the search returned)
STEP 4: A separate judge request: for each citation, "does the paper study exactly these entities and this relationship?" → yes/no
Steps 1–3 are one agent loop. Step 4 is a separate request, ideally in a different model.
Example
Task: You run content for a supplement brand on Amazon. A marketer brings you a product listing: "Collagen capsules restore joint cartilage in people over 40." You need to know whether the claim has any backing in the literature — without getting pretty but empty citations from the model.
Prompt (for an agent with search access: Claude with web search, ChatGPT Search, Perplexity, etc.):
You are an assistant for verifying medical claims. Decide whether the scientific literature SUPPORTS the claim (SUPPORT), CONTRADICTS it (CONTRADICT), or there is NOT ENOUGH INFORMATION (NEI). Use NEI if the evidence you find doesn't directly address the claim.
Claim: "Oral collagen restores joint cartilage in people over 40."
You can search PubMed and scientific sources. Search works by keywords (terms are joined with AND), so short queries of 2–6 key terms work best. You can use OR. Don't paste the whole claim in as a query. Write the queries yourself: extract the entities (substance, condition, population) and fix typos. You have a maximum of 4 searches.
At each step, choose: either a new search query or an answer.
Base your verdict only on the papers you found. Cite only papers the search actually returned, and never name papers "from memory."
Response format — JSON:
{"label": "...", "cited_pmids": [...], "rationale": "..."}
Result: The model will run a few searches with short term-based queries (substance, joint, age). Then it outputs JSON with a verdict, a list of the papers it found, and a rationale. All citations should come from the search results. If there's no direct evidence, expect NEI rather than a confident "yes" or "no." With a second request (template below), run the citations through a relevance check, and open 2–3 sources yourself before publishing.
Why It Works
The weakness. If you feed the whole claim into search, the keywords don't match the text of the papers. A typo in a substance name kills the match. Without search, the model recalls plausible-looking paper IDs: the format is right, the IDs exist, but there's no connection to the claim. The model guesses what a citation looks like; it doesn't remember what's in it.
The strength. The model is good at two things. First, turning a long sentence into a short query of entities and fixing typos (that's what tripled search success). Second, reasoning over text that's right in front of it. With the right papers, accuracy reached 79–86%. The model's reasoning is fine; it's the material that's missing.
How the method uses this. It hands the model the part where it's strong: writing the query. It forbids the part where it lies: citing from memory. The "cite only what you found" rule nearly eliminated "outside" citations for agents (0–2.6%). A separate relevance check catches what an existence check can't see.
Levers: - Search limit (4 in the study). Agents used an average of 1.4–1.7 out of 4. Raising the limit is pointless until you make them use it. - Query rules ("2–6 terms, OR, field tags"): tune them to your source: Google Scholar, the Census Bureau, your own database. - NEI strictness. The agent says "not enough information" less often than it should (33–42% correct NEI versus 72–81% for brute-force search). If a false "supported" costs you more, strengthen the NEI requirement. - Judge criterion. "Only if the paper studies the specific entities and relationship" is strict. Loosen it and you'll let through more "close to the topic" papers.
Prompt Template
These are the prompts from the study's appendix.
Search agent:
You are an assistant for verifying scientific claims in the field of {field}.
Decide whether the literature SUPPORTS the claim (SUPPORT), CONTRADICTS it (CONTRADICT), or there is NOT ENOUGH INFORMATION (NEI). Use NEI if the evidence you find doesn't directly address the claim.
Claim: "{claim}"
You can search {search_source}. Search works by keywords (terms are joined with AND), so short queries of 2–6 key terms work best. You can use OR{and_field_tags}. You have a maximum of {number} searches.
At each step, choose one: a search query or an answer.
Base your verdict only on the papers you found. Cite only what the search actually returned.
Response format — JSON:
{"label": "SUPPORT | CONTRADICT | NEI", "cited_pmids": [...], "rationale": "..."}
Citation check (a separate request, ideally in a different model):
Claim: "{claim}"
Cited paper: {title_and_abstract}
Answer "yes" only if the paper studies exactly the entities and the relationship described in the claim (that is, it could plausibly support or refute it). Answer "no" if it's off-topic or only loosely related.
What to fill in: {field} — medicine, nutrition science, finance, etc. {search_source} — the tool the agent has. {number} — the search limit (4 in the study). {claim} — a single checkable statement. For the citation check, take the title and abstract from the search results, not from the model's answer.
🚀 Quick start — paste into the chat:
Here's a template for a claim-verification agent and a citation check. Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what search tool you have, how it handles queries (AND/OR, language, tags), what your field is, and which costs you more: missing an error or playing it too safe. That way the query rules match your source. It'll take the rest from the template.
Limitations
⚠️ A pilot study: on 100 claims, none of the differences between the realistic options (agent, brute-force search, no search) reached statistical significance. The direction was the same for both models, but it's a trend, not a proven fact.
⚠️ Small models from one provider: only fast lightweight models were tested. Results may differ on more powerful models and other model families.
⚠️ The agent still finds little: the right paper was found in only ~20% of cases. The 3x gain is meaningful as a ratio, but the absolute level is low.
⚠️ The agent is lazy about searching: it barely searches beyond the first query and says "not enough information" less often than it should. The authors suggest "forcing a minimum number of searches," but didn't test it.
⚠️ The relevance judge is a single model: it was validated on only 50 control examples (it discriminated well, but the sample is small). It's no substitute for human review of critical citations.
⚠️ "Errors" aren't always errors: the dataset labels are tied to specific abstracts, while the agent searched all of PubMed as of 2026. Some "misses" may have been newer evidence.
How They Tested It
The author used the SciFact dataset: scientific claims labeled by experts as "supports," "refutes," or "not enough information." They randomly picked 100 claims and ran them through four verification setups on two lightweight Gemini models, with searches running against the real PubMed. The model, answer format, and temperature were identical; only the "harness" around the model changed. That made it possible to separate the effect of search from the effect of the model itself.
The four modes: no search, a single query made from the claim itself, an agent writing its own queries (up to 4), and an "oracle" given the correct papers. The oracle served as the ceiling: it showed what the models can do with perfect search. Citations were checked separately: does the cited ID exist, does it match the reference paper, and is it relevant to the claim. Relevance was judged by a model from a different family (Qwen). It was validated on controls: it rated 21 of 25 correct papers as relevant and 0 of 25 decoys.
The main results. Brute-force search found the right paper in only 6% of cases and didn't improve accuracy. It turned confident errors into "not enough information." The agent tripled the hit rate (to 19–20%) and gave the best accuracy (62%), but the difference isn't significant. The oracle gave 79–86%, so that 17–24 point gap is significant, and the bottleneck really is search. The most striking result: 3 relevant out of 327 existing citations for the no-search model versus ~67% for the agent. The practical takeaway: "the link exists" is not a safeguard.
Adaptations and Extrapolations
🔧 Technique: require a minimum of evidence before answering → fewer lazy answers after a single search
The authors propose this step as an idea (untested). Add to the agent's prompt:
Don't answer after the first search. Run at least 2 different queries (for example, using another name for the substance or a synonym), and answer only if you've found at least one paper that directly addresses the claim. If there's no such paper after all searches, answer NEI.
You'll need to test the effect on your own sample: compare the share of correct NEI answers before and after.
🔧 Technique: a judge filter on the output → only relevant citations make it into the report
An extrapolation of the principle: before submitting a report, run each citation through a relevance check and keep only the "yes" ones:
For each citation in the list below, answer "yes" or "no" by this criterion: the paper studies exactly the entities and the relationship in the claim. Return only the citations marked "yes"; for the rest, explain why they don't fit.
Claim: "{claim}"
Citations: {list_of_titles_and_abstracts}
This carries the study's judge idea over to any report with citations: market reviews, legal memos. The effect outside biomedicine wasn't tested in the paper.
Resources
- Paper: Real PMIDs, Wrong Papers: Harness Design for Biomedical Claim-Verification Agents — Mohammed Saqlain (independent researcher, Bengaluru).
- Code, prompts, and call cache: github.com/saqlain2204/real-pmids-wrong-papers
- Dataset: SciFact (Wadden et al., 2020).
- Related work: ReAct (Yao et al., 2023), PaperQA/PaperQA2 (Lála et al., 2023; Skarlinski et al., 2024), MedRAG (Xiong et al., 2024), GeneGPT (Jin et al., 2024), an analysis of ChatGPT citation fabrication (Walters & Wilder, 2023).