TL;DR
SEA is a technique from the FAIR-RAG framework that turns checking whether you have enough information into an analytical checklist. The LLM breaks the question down into a list of required facts, checks them against the evidence collected, and explicitly names the gaps — what still needs to be found out. Those gaps become the basis for new, targeted questions.
LLMs struggle with complex multi-hop questions (for example: "Compare the architectural style of the building where the Mona Lisa hangs and the one that houses the Rosetta Stone"). The standard approach — a single retrieval attempt — produces an incomplete answer. The model either finds facts about only one of the objects or skips half the question entirely. The reason: there's no mechanism to explicitly track what's already known and what isn't yet.
SEA solves this by role-playing an intelligence analyst. The LLM gets the role of a Strategic Intelligence Analyst with a mission: (1) break the question down into Required Findings (what needs to be found out), (2) check the Confirmed Findings (what's already known from the evidence), (3) identify the Remaining Gaps (what's still unknown). If there are gaps, the loop repeats with new clarifying questions aimed squarely at those gaps. Once every item on the checklist is closed, the model gives the final answer.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Decompose the query
The LLM breaks the complex question into 2-4 independent sub-questions → a set of sub-queries
STEP 2: Retrieve and filter (iteration 1)
Find information for each sub-query → collect evidence
STEP 3: SEA — gap analysis
The LLM as Strategic Intelligence Analyst:
- Checklist of Required Findings (what's needed)
- Check of Confirmed Findings (what we have)
- List of Remaining Gaps (what's missing)
→ Decision: "Enough" (Yes) or "Not enough" (No) + what exactly is missing
STEP 4: If "No" — clarifying questions
Generate new targeted queries ONLY for the gaps → go back to Step 2
STEP 5: If "Yes" — final answer
Strictly based on the collected evidence, with source citations
The Step 2-4 loop repeats at most 3 timesExample in Action
⚠️ Method limits: SEA works for multi-step analytical tasks where you need to gather facts from different places and pull them together. It's not for simple factual questions ("the capital of France") or creative tasks.
Task: You're launching an online English school for software engineers. You need to figure out which successful EdTech startups have done something similar, how they landed their first customers, and what from their experience you can apply to your niche.
Prompt (iteration 1):
I need to research successful EdTech startups that served narrow professional niches.
Break this question into 2-3 sub-questions for research:
- Which EdTech projects succeeded in niche B2C education
- How they attracted their first paying customers
- Which of their strategies could apply to an online English school for software engineers
Now find information on these sub-questions. Then act as a Strategic Intelligence Analyst:
1. Required Findings — what I want to find out (a checklist of facts)
2. Confirmed Findings — which of the Required items you've already found
3. Remaining Gaps — what's still unknown
If there are Remaining Gaps — say "NO, need more detail" and list the gaps.
If all Required Findings are confirmed — say "YES, enough".Result: The model will do an initial search (say, finding Duolingo, Preply, Codecademy), followed by a structured analysis:
- Required Findings: examples of niche EdTech / their acquisition channels / applicability to English for engineers
- Confirmed Findings: startup names, general strategies
- Remaining Gaps: concrete numbers for the first customers, launch details specific to language schools
The model will answer: "NO, need more detail. Gap: the exact mechanics of acquiring the first 100 customers in language EdTech."
Prompt (iteration 2 — closing the gaps):
You identified these gaps:
- The exact mechanics of acquiring the first 100 customers in language EdTech
Now find information ONLY on these gaps. Focus on how Preply or other online language schools got started.
Apply SEA again:
1. Required Findings (the same as before)
2. Confirmed Findings (everything known NOW, including data from the previous iteration)
3. Remaining Gaps (what's still needed)
If there are no gaps — YES, give the final synthesis.Result: The model will find cases from Preply's early days (search ads, free trial lessons, partnerships with developer communities), add them to Confirmed Findings, and check the list. If everything is closed, it'll say "YES, enough" and give a final answer with concrete strategies and numbers you can apply to your English school for engineers.
Why It Works
LLMs are good at local generation — answering a specific question. But they're bad at tracking multi-step completeness — keeping every part of a complex request in mind and checking that nothing got missed. Without explicit structure, the model can get carried away with one branch (say, a detailed story about Duolingo) and forget the other (how it applies to your niche).
LLMs are excellent role performers when given clear instructions. The Strategic Intelligence Analyst role with the "Required → Confirmed → Gaps" format gives the model an explicit thinking procedure. It can't "forget" part of the question, because the Required Findings checklist is always in front of it.
SEA uses the LLM's strength (following role instructions and structuring information) to compensate for its weakness (tracking completeness in multi-step tasks). The checklist turns the abstract check "is there enough information?" into a concrete list of "what we have, what we don't" — that's the key insight of the method.
Control levers:
- Number of iterations (1-3): one is enough for simple tasks; use 3 for deep research. Beyond 3, quality drops as noise piles up.
- Level of detail in Required Findings: you can give an explicit list of what you want to learn in the first prompt, or let the LLM break the question down itself.
- Strictness of the check: add the instruction "treat even partially missing data as a gap" for a more thorough search.
- Gap format: you can ask it to rank gaps by importance, so the next iteration only hunts for the critical ones.
Prompt Template
I need to {describe a complex task that requires several facts/sources}.
Step 1: Break the task down into 2-4 key sub-questions for research.
Step 2: Find information on these sub-questions.
Step 3: Act as a Strategic Intelligence Analyst. Run a Structured Evidence Assessment (SEA):
1. **Required Findings** — a checklist: what needs to be known to answer my task
2. **Confirmed Findings** — which Required items are already confirmed by the evidence found
3. **Remaining Gaps** — which Required items are still unknown
If there are Remaining Gaps — answer "NO" and list the gaps.
If all Required items are confirmed — answer "YES".
[If the answer is NO — continue:]
Step 4: Now find information ONLY on the identified gaps: {gaps from the previous step}.
Step 5: Apply SEA again with the same Required Findings checklist. Add the new data to Confirmed Findings.
Repeat Steps 4-5 at most 2 times.
[When the answer is YES:]
Step 6: Give the final answer STRICTLY based on the Confirmed Findings. Back up every fact with a source.What to fill in:
{describe a complex task}— your multi-step question (a comparison, an analysis, a piece of research){gaps}— the list of Remaining Gaps from the previous SEA iteration
🚀 Quick start — paste into the chat:
Here's an SEA template for complex multi-step tasks. Adapt it to my task: [your task].
Ask me questions so you can fill in the Required Findings correctly.
[paste the template above]The LLM will ask about the details of your task and build a Required Findings checklist tailored to your request. It'll take the Strategic Intelligence Analyst role pattern and the "Required → Confirmed → Gaps" structure from the template and apply them to your case.
Limitations
⚠️ Multiple rounds are a barrier to entry: The method takes 2-3 rounds of interaction with the LLM. If you need a quick answer to a simple question, SEA is overkill. Use it for tasks where accuracy matters more than speed.
⚠️ Checklist quality depends on the model: Weaker models may produce an incomplete or fuzzy list of Required Findings. For SEA it's better to use top-tier models (GPT-4, Claude Opus, Gemini Pro).
⚠️ Not for creative work: SEA is about factual completeness, not idea generation. It's not a fit for brainstorming, storytelling, or copywriting.
How They Tested It
The team built FAIR-RAG — a full RAG pipeline with code, vector databases, and APIs. They tested it on 4 benchmarks (HotpotQA, 2WikiMultiHopQA, MusiQue, TriviaQA) with 1000 questions each, comparing it against 7 strong baselines: standard RAG, Iter-Retgen, Self-RAG, Adaptive-RAG, and others.
Key insight: SEA with explicit gap analysis proved critical precisely on multi-hop tasks. On HotpotQA, FAIR-RAG scored F1 = 0.453 versus 0.370 for the best baseline (Iter-Retgen) — a gain of 8.3 points. That's a huge jump for benchmarks like these.
Why this result? The researchers found that methods without explicit gap analysis (for example, Iter-Retgen simply uses the entire previous answer for the next search) either repeat what's already known or wander off course. SEA with a checklist forces the model to search specifically for the missing facts, without wasting iterations on noise.
An important observation: the sweet spot is 2-3 iterations. The first iteration lays the foundation, the second closes the main gaps, and the third is fine-tuning. A fourth already makes results worse — irrelevant information piles up. On TriviaQA (simple questions), any iteration after the first only hurts — confirmation that SEA is for complex tasks, not factoids.
They also ran a component analysis: SEA was 72% accurate on HotpotQA and 82-83% on MusiQue and 2WikiMultiHopQA at judging whether the evidence was sufficient. That's a high-stakes decision (keep searching or not), and the model handles it reliably exactly where it matters — on multi-hop tasks.
Original from the Paper (optional)
Context: The researchers used an LLM in the role of a Strategic Intelligence Analyst for the SEA module. Here's an example prompt from the original:
You are a Strategic Intelligence Analyst.
Your mission is to determine if the provided evidence is sufficient to answer the user's query comprehensively and accurately.
Follow this structured thinking process:
1. **Deconstruct the Query**: Break down the user's question into a checklist of discrete, required informational components or "findings."
2. **Audit the Evidence**: Systematically evaluate the collected evidence against your checklist.
- **Confirmed Findings**: Which required components are explicitly supported by the evidence?
- **Remaining Gaps**: Which required components are NOT addressed or only partially addressed?
3. **Sufficiency Decision**:
- If ALL required findings are confirmed by the evidence → "Yes"
- If ANY required finding remains a gap → "No"
You MUST output your assessment in this EXACT format:
**Required Findings:**
[Numbered list of informational components needed]
**Confirmed Findings:**
[Numbered list of findings confirmed by evidence]
**Remaining Gaps:**
[Numbered list of findings NOT confirmed]
**Is Sufficient:** [Yes/No]
Be rigorous. Do not assume. If a finding is not explicitly supported, it is a gap.Adaptations and Extrapolations
💡 Adaptation for stress-testing your own arguments:
Use the SEA checklist to check your own writing or presentations. Instead of searching for outside evidence, you check how complete your arguments are.
Here's my article/pitch: {text}
Act as a Strategic Intelligence Analyst.
Required Findings — what the text must contain for the reader to be convinced:
- The problem is clearly stated
- The solution is concrete
- There's evidence it's viable
- A call to action
Confirmed Findings — which of these are in the text
Remaining Gaps — what's missing
If the Gaps aren't empty — suggest how to strengthen the text.🔧 Technique: Process transparency → trust in the result
Add to the prompt: "Show the intermediate SEA iterations — the full checklist, the gaps, the new queries." This helps you:
- See the LLM's train of thought
- Catch mistakes early
- Learn to structure complex queries yourself
💡 Combo: SEA + personas instead of a faceless analyst
Instead of an abstract Strategic Intelligence Analyst, assign specific roles with different focuses:
Iteration 1:
You are an Optimistic Marketer. Find evidence of MARKET OPPORTUNITIES for EdTech teaching English to software engineers.
SEA checklist: market size, willingness to pay, competition
→ Remaining Gaps: retention data for niche schools
Iteration 2:
You are a Skeptical CFO. Find evidence of the RISKS: why it might not take off.
SEA checklist: barriers to entry, unit economics, churn rate
→ Add to the Confirmed Findings from iteration 1
Result: The full picture of opportunities AND risks.Personas with clear roles deliver sharper criticism and different angles. The SEA structure keeps you from losing focus.
Resources
FAIR-RAG: Faithful Adaptive Iterative Refinement for Retrieval-Augmented Generation
Authors: Mohammad Aghajani Asl (Sharif University of Technology), Majid Asgari-Bidhendi (Iran University of Science and Technology), Behrooz Minaei-Bidgoli (Iran University of Science and Technology)