TL;DR
Agentic XAI is a technique where an LLM improves its own explanation over several rounds of self-review. The model gets data or an analysis result, produces an explanation, then asks itself "what could be better?", generates extra charts or calculations, and rewrites the explanation with the new information. The researchers ran 11 rounds of refinement (0-10) on the task of explaining rice yield predictions.
The headline finding: explanation quality improves only for the first 3-4 rounds, then drops sharply. At its peak the model gives recommendations that are 30-33% better than at the start. But after 4-5 rounds degradation kicks in: explanations get watery, drift into abstractions and lose focus. It's an analog of the bias-variance trade-off: early rounds are too simple (they miss details), late rounds are too wordy (the point drowns in filler). Both expert agronomists (n=12) and 14 different LLMs rated the outputs, and both groups showed the same pattern.
The method runs as a loop: the LLM reviews its previous answer → spots gaps → writes code for additional analysis → gets new data → rewrites the explanation. After 3-4 cycles you need early stopping, because further iterations hurt. The principle is universal: it works not just for XAI but for any complex explanation or analysis.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
ROUND 0: Baseline explanation
The LLM gets data/a chart → produces a first explanation
ROUNDS 1-10: Refinement loop (each round = a separate request)
STEP 1: Gap analysis
→ "What additional data would improve this explanation?"
→ Generate Python code for calculations/charts
STEP 2: Get the data
→ Run the code, get new charts/statistics
STEP 3: Update the explanation
→ "Rewrite the explanation using the new information"
→ The LLM synthesizes old + new
RULE: Stop after 3-4 rounds (quality peak)
In a chat context: It works within a single conversation: the history is kept, and each round builds on the previous ones.
Example
Task: You're launching a product feature in a subscription service. You look at a month of conversion data: some user segments convert 3x better than others. You need to understand why and what to do about it.
Round 0 — Baseline explanation:
Here is a month of subscription conversion data:
[paste a table or a description of the segments]
Explain why conversion differs and give recommendations.
Round 1 — First improvement:
What additional data or calculations would help improve this explanation?
Suggest what to calculate or which charts to build.
Round 2 — Running the analysis:
[Take the suggested calculations and do them by hand, or ask the LLM for code]
Here are the results:
[paste the new data]
Rewrite the recommendations using this information.
Rounds 3-4 — Repeat the loop:
Repeat steps 1-2 one or two more times.
Stop at round 4: after that, quality goes downhill.
Result:
By round 3-4 you'll have an explanation with specific numbers, cause-and-effect links (not just "segment A is better" but "segment A converts better because it comes through channel X, where motivation is higher"), and prioritized actions ("focus on channel X, because ROI there is 2.5x higher"). The answer will be more detailed and sharper than the first version, but it won't drown in filler the way it does after 7-8 rounds.
Why It Works
The LLM's weakness: In its first answer the model relies only on the data in front of it. It doesn't know which additional calculations are needed for the full picture: correlations, subgroups, time patterns. The first answer is often shallow because the model is working with limited context.
The LLM's strength: The model is great at analyzing its own gaps and generating hypotheses when asked. It can suggest "calculate the correlation between X and Y" or "plot the weekly trend" if you prompt it with "what would improve the explanation?". And it's good at synthesizing heterogeneous data into one explanation when it receives it gradually.
How the method uses this: The iterative loop turns a weakness into a strength. Instead of demanding a full analysis up front (the model doesn't know what's needed), we let it reflect: "what did you miss?". The model proposes calculations → gets the data → deepens the explanation. The first 3-4 rounds fill real gaps. But after that reward overoptimization sets in: the model tries to improve where there's nothing left to improve, adds abstractions and filler, and loses specifics.
Control levers:
- Number of rounds: cut it to 2-3 for simple tasks (saves time), raise it to 4-5 for complex ones (but no more, or quality drops)
- A "no fluff" instruction: in later rounds add "keep it specific, no generic statements" to reduce verbosity
- Type of additional analysis: steer the model explicitly: "suggest only quantitative metrics" or "only qualitative insights"
- Exit condition: replace the fixed number of rounds with a criterion: "stop when a new round adds no new facts"
Prompt Template
Basic version (3-4 rounds)
Round 0:
Here is the data/analysis result:
{data_or_description}
Explain {what_to_explain} and give recommendations.
Rounds 1-3 (repeat 2-3 times):
What additional data, calculations or charts would help improve this explanation?
Suggest specific questions to ask of the data.
↓ Run the suggested analysis ↓
Here are the results:
{new_data}
Rewrite the recommendations using this information.
Keep it specific, avoid generic phrases.
Placeholders:
- {data_or_description} — a table, a chart, a text description of the situation
- {what_to_explain} — "why the metric went up", "what drives the result", "which factors matter"
- {new_data} — the results of the calculations/charts from the previous step
Important: Stop after 3-4 rounds. Quality drops after that.
Extended version (with an explicit quality metric)
For complex tasks where you need to keep degradation in check:
ROUND {N}:
Current explanation:
{current_explanation}
Rate it on these criteria:
1. Specificity (numbers, examples, no generic statements)
2. Completeness (the important factors are covered)
3. Conciseness (no filler)
Then:
- If completeness < 7/10: suggest additional analysis
- If specificity or conciseness < 6/10: rewrite without additions, tighten it
- If everything is > 7/10: the explanation is done, stop
This adds an exit condition instead of a fixed number of rounds.
🚀 Quick start — paste this into the chat:
Help me adapt the iterative explanation-refinement technique to my task.
My task: [describe what needs explaining: the data, the situation, the problem]
Use this template:
[paste the basic version of the template above]
Ask clarifying questions so you can fill in the placeholders correctly.
The LLM will ask what exactly to explain, what data is available and what output format you need, because the technique requires a clear definition of "what we're improving". It will take the "analysis → additional data → update" pattern and adapt it to your task.
Limitations
⚠️ Degradation after 4-5 rounds: Explanation quality drops sharply, by 60-80% from the peak. The model starts adding filler and abstractions and loses specifics. It's not a bug but a fundamental property: reward overoptimization. Don't try to improve forever.
⚠️ Not for quick answers: The method needs 3-4 LLM requests plus running intermediate analysis. If you need a fast answer, use a regular prompt, not iterations.
⚠️ Risk of drifting off the point: In later rounds the model may wander into theory instead of practical recommendations. Explicitly add "stay focused on {specific_task}" to your prompts after round 3.
⚠️ You need access to data between rounds: If the model suggests calculations you can't run (no data, no tools), the loop breaks. It works best when you have access to the raw data or can request additional information.
How They Studied It
The researchers took real farm field data from Japan: 26 fields, 3 years, rice yields plus 37 variables covering soil, weather, varieties and farming practices. They trained a Random Forest to predict yield, then SHAP (a technique for explaining ML predictions) produced a chart of "which factors matter". That chart was fed to Claude Sonnet 4 with the prompt "explain to farmers what affects the yield".
Then came the iterative loop: the model looked at its previous answer, suggested "which chart or calculation would improve the explanation", generated Python code for the analysis, got new data and rewrote the recommendations. They repeated this for 11 rounds (0-10). Each round kept the history, so the model remembered all previous data and conclusions.
Then they rated all 11 versions of the recommendations blind. The order was randomized so nobody knew "this is round 0" or "this is round 10". The raters were 12 agronomists (PhDs, members of the Japanese agronomy society) and 14 LLMs (GPT-4o, GPT o1, Claude Opus/Sonnet, Grok, DeepSeek, Gemini: top models from 5 companies). Each scored the outputs on 7 criteria: specificity, clarity, conciseness, practicality, contextual relevance, cost awareness and scientific soundness.
The results were striking: both groups of raters drew an inverted U-curve. Quality rose until round 3-4 (+30-33% over the start), then collapsed. Why? A detailed analysis showed that early rounds added specifics and depth (correlations, time effects, targeted recommendations). But after round 4 the model began bloating the explanations: more words, less new info, a slide into abstractions like "a comprehensive approach" instead of "increase the nitrogen dose by 15%". It's classic distribution drift: the model optimizes a metric (improve the explanation), but the metric stops correlating with real quality.
The researchers confirmed a bias-variance trade-off for explanations: simplicity = missed details (bias), complexity = focus lost in filler (variance). The sweet spot is in the middle.
Adaptations and Extrapolations
🔧 Technique: Explicit quality metric → degradation control
Add a self-assessment to each round so the model can tell when the improvements have run out:
After each update of the explanation:
Rate it on a 1-10 scale:
- Specificity (facts, numbers, no generic statements)
- Completeness (the important factors are covered)
- Conciseness (no filler or repetition)
If specificity or conciseness dropped since the last round → STOP, return the previous version.
If completeness didn't improve → STOP, you've hit the limit.
This turns a fixed early stop (4 rounds) into an adaptive one: the model senses degradation on its own.
🔧 Technique: Splitting roles → less verbosity
Use two agents: one improves, the other trims:
ANALYST AGENT:
Suggest additional analysis and rewrite the explanation.
EDITOR AGENT:
Review the updated explanation:
- Remove repetition
- Replace generic phrases with specific facts
- If the new round added no facts → return the previous version
Final output = the editor's version
This fights reward overoptimization: the editor blocks text bloat that brings no new information.
🔧 Combining with Chain-of-Thought: iteratively deepening reasoning
Apply the principle not to explanations but to solving a hard problem:
ROUND 0:
Here is the problem: {description}
Propose a solution with a rationale.
ROUNDS 1-3:
Look at your solution critically.
Which aspects of the problem did you miss?
What data/calculations would help verify the solution?
[You get the additional analysis]
Update the solution. Keep it specific, no filler.
STOP after round 3-4.
Example: Choosing between two career options. Round 0 = a gut-feel choice. Rounds 1-3 = the model asks for data (finances, schedule, growth, risks), runs the numbers and deepens the rationale. By round 3 you get a decision backed by numbers and scenarios, not just "option A is better".
Resources
Agentic Explainable Artificial Intelligence (Agentic XAI) Approach To Explore Better Explanation — Tomoaki Yamaguchi, Yutong Zhou, Masahiro Ryo, Keisuke Katsura (Gifu University, ZALF, Brandenburg University of Technology, Kyoto University)
Reproducibility materials: https://doi.org/10.5281/zenodo.17876330
Related concepts: ReAct (Yao et al., 2023), Self-Refine (Madaan et al., 2023), LLM-as-judge (Zheng et al., 2023)