Grounded prompting and RAG: getting answers from your documents, not the model's memory

What grounded prompting is

Grounded prompting makes the model answer from material you supply (a document, retrieved passages, a data dictionary) rather than from what it remembers. The prompt says what the source is, how to use it and what to do when the answer is not in it. Retrieval-augmented generation (RAG) is the same idea with a search step in front.

The techniques fall into three groups:

• Framing the source: "based ONLY on the document below", plus an explicit line for when the answer is missing.
• Anchoring the output to the text: verbatim quotes, numbered lines, claim IDs, citations after every point.
• Checking coverage: what the evidence confirms, what is still missing, and which sources to trust.

Why it works: from recall to extraction

A pasted document does not switch off the model's memory. The three-regimes study shows the model silently weighs your text against what it learned, holds on harder for famous facts, and resists more if it has already answered the question earlier in the chat. One framing line changes the task itself from recall to extraction: with "Based ONLY on the following document", Claude followed the document 100% of the time; with "use your own knowledge, the document may contain errors", 7.6%. Across models the swing reached 29-94 percentage points.

The SafeLLM study pushes the idea further. Writing an answer from a document is generation, which blends source and memory; choosing lines is classification. Numbering the document's lines and asking only for line numbers means the answer is assembled from original text, and this LINES strategy reached up to 95% agreement on medical terms in NHS clinical guidelines. Meaning works the same way for data: in the semantic-layer study, a 1-2 page markdown file defining metrics and table conventions raised three frontier models from 45-50% to 68% on the same 100 SQL questions, and the gap between models was negligible next to the gap between "with context" and "without".

Variants that tested well

1. Gap analysis before the answer. Structured Evidence Assessment makes the model list Required Findings, Confirmed Findings and Remaining Gaps, then search only for the gaps. In the FAIR-RAG pipeline it reached F1 0.453 on HotpotQA against 0.370 for the best baseline; 2-3 iterations was the sweet spot, and a fourth added noise.
2. Read the documents clean, then reason. Reasoning models used retrieved documents worse the longer they had reasoned before the search, even when the documents held the answer. SAKE places the documents twice, once at the start of the prompt and again after each reasoning step, and reports up to +37.6% F1 on hard bridge-comparison questions.
3. Claims with quotes, then the summary. CAMS extracts atomic claims with verbatim quotes from each source, then writes the summary only from those claim IDs, stating contradictions between sources instead of smoothing them over.
4. Three commands for document analysis. The structured document-analysis prompt separates instructions from data with tags, asks for an analysis plan, a citation after every point, and a plain "the text has no information about X" instead of filling gaps.
5. Check each source alone before synthesis. The epistemic blind-spot study found models trust academic-looking style over the numbers when sources compete; evaluating each source in isolation first caught implausible statistics such as a 0.97 correlation from n=12.

When it helps and when sources mislead

Grounding helps most when the model already "knows" the topic (law, medicine, well-known companies) and you need your version of it, and when exact wording carries risk: contracts, policies, protocols. It is weaker for synthesis questions that join distant parts of a document; the SafeLLM authors note that line selection picks lines but does not connect them.

Three papers show sources misleading the model. The GroupQA study found that one argument paraphrased five times changed answers more often than five independent arguments (76.5% vs 67.6%), that the first documents in the list carried more weight than the last, and that models asked which document mattered most were wrong 74% of the time; chain-of-thought shifted results by less than 0.5%. The AuthorityBench study found that attaching a citation to a claim raised error rates across all seven models, worst for a true claim with a fabricated source, with up to 77% wrong answers in some conditions; "According to X, ..." at the start of a sentence was among the riskiest formats and "(Author, year)" at the end the safest. And in the over-searching study, search access raised accuracy on answerable questions by 12% but lowered correct abstention by 7%, doubling tokens per correct answer from 382 to 765.

So grounding is more than adding documents: frame the source, interleave conflicting documents instead of grouping them by side, tell the model to weigh quality over repetition, and give it an explicit way to say the answer is not there (see abstention prompts and lost in the middle).

Based ONLY on the document below, answer the question. 
Do not use prior knowledge. Use only what 
the document says. If the answer isn't in the document, 
say so directly.

Document:
{document_text}

Question: {question}

TEMPLATE (two requests):

── REQUEST 1 ──
Number every line of the text below.
Numbering format: [N] line text
Number every line, including short ones.

{paste the document or section text}

── REQUEST 2 (after you get the numbered text) ──
Here is the numbered text.
Question: {your question}

Instructions:
- Return only the numbers of the lines needed to answer
- Pick the minimal sufficient set
- No text of your own — numbers only
- Format: comma-separated numbers

{paste the numbered text from request 1}

You are a Strategic Intelligence Analyst.

Your mission is to determine if the provided evidence is sufficient to answer the user's query comprehensively and accurately.

Follow this structured thinking process:

1. **Deconstruct the Query**: Break down the user's question into a checklist of discrete, required informational components or "findings."

2. **Audit the Evidence**: Systematically evaluate the collected evidence against your checklist.
   - **Confirmed Findings**: Which required components are explicitly supported by the evidence?
   - **Remaining Gaps**: Which required components are NOT addressed or only partially addressed?

3. **Sufficiency Decision**:
   - If ALL required findings are confirmed by the evidence → "Yes"
   - If ANY required finding remains a gap → "No"

You MUST output your assessment in this EXACT format:

**Required Findings:**
[Numbered list of informational components needed]

**Confirmed Findings:**
[Numbered list of findings confirmed by evidence]

**Remaining Gaps:**
[Numbered list of findings NOT confirmed]

**Is Sufficient:** [Yes/No]

Be rigorous. Do not assume. If a finding is not explicitly supported, it is a gap.