TL;DR
Evolving Agent Teams is a setup for long-running work with coding agents. A team of agents is overseen by a lead supervisor agent. It tracks progress, and when the work stalls, it makes a diagnosis and restructures the team: it invents new roles, changes the workflow and the rules for recording knowledge. All findings, failures and decisions pile up in a shared folder of files. Each new team starts not from scratch but from the accumulated experience.
The key finding: a strong agent almost always gets stuck at a local optimum, and a more powerful model doesn't cure that. At first it makes fast progress. Then it picks the path that's easier to implement but has a low ceiling. It burns the rest of the budget on minor polishing. When a hard but promising path breaks, the agent abandons it and never comes back. A fixed team of four roles hits a ceiling too: the phases of work change (correctness → debugging → profiling → redesign), but the roles stay the same.
The method boils down to three axes the supervisor adjusts after every plateau: roles (who is responsible for what), coordination (pipeline or parallel branches) and knowledge management (where to write what, and what must be read). It doesn't pick roles from a list; it invents them for the current breakdown: “Profiler,” “Skeptic,” “Knowledge Integrator.”
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
LOOP (runs when the team has finished its work OR on a timer):
STEP 1: Check progress → metric, log, team files
STEP 2: Diagnose the plateau → one of the patterns:
• drift toward the easy path
• loss of discovered insights
• stuck in debugging
• micro-optimization without gains
STEP 3: Restructure the team along three axes T = (Roles, Coordination, Knowledge)
• Roles → new narrow roles for the current breakdown
• Coordination → pipeline / parallel branches / both
• Knowledge → where to write, what must be read
STEP 4: Launch team v+1 → it reads the files of previous teams
STEP 5: Record in the history: diagnosis, decision, result → back to STEP 1
SHARED FOLDER (persists the whole time):
learnings — accumulated empirical observations
artifacts — results and notes from every team and role
evolution — the supervisor's reasoning and decisions per version
The steps are carried out by the supervisor as a separate agent, not by a single prompt. The worker agents run as separate sessions.
Example
The method shines where the task is long, open-ended, with a measurable metric and a correctness check. The authors' GPU kernels are a special case. Speeding up a service is closer to home for most readers.
Task: The team behind an online store with marketplace-level traffic asks Claude Code to speed up catalog search. In an hour, the agent cut p95 latency from 900 to 520 ms by adding a cache. Then it spends three hours polishing trivia and gains nothing. The idea of rebuilding the index broke the tests, and the agent gave up on it.
Prompt (to the supervisor):
You are the supervisor of an agent team. The team's task: reduce the p95 latency
of catalog search for an online store (currently 520 ms, target 200 ms or less)
without breaking the tests in /tests/search.
Your job is not to write code, but to monitor the team and restructure it.
KNOWLEDGE FOLDER (read everything before every decision):
- /knowledge/learnings.md — observations confirmed by measurements
- /knowledge/artifacts/ — results and notes from previous teams
- /knowledge/evolution.md — your past diagnoses and decisions
LOOP:
1. Check progress: metric by version, latest commits, agent logs.
2. Make a diagnosis. Look for patterns:
- drift toward the easy path (a hard idea dropped after the first failures);
- loss of insights (a finding from a past version isn't being used);
- stuck in debugging (one breakage blocks all the work);
- micro-optimization (lots of effort, less than 2% gain).
3. Restructure the team along three axes:
- ROLES: invent narrow roles for the diagnosis (not from a ready-made list).
Explain why a single agent can't combine these tasks.
- COORDINATION: pipeline, parallel branches, or both. If an agent
is debugging a blocking problem, it doesn't touch the main branch.
- KNOWLEDGE: specify where each agent writes its results and which document
it must read before starting work.
4. Write to evolution.md: what you observed → diagnosis → what you changed → expectation.
5. Update learnings.md only with facts confirmed by measurements.
Output: the diagnosis, the new team lineup with each role's mandate,
the coordination scheme, the knowledge-recording rules.
Result: The supervisor will deliver a diagnosis (for example, “the index-rebuild idea was abandoned because of broken tests”). Then it will propose a team lineup with each role's mandate, for example a debugger on a separate branch plus a profiler. It will lay out the coordination scheme and the knowledge-recording rules. It will also update the decision log. The exact team lineup depends on your codebase.
Why It Works
Weakness. A single agent splits its limited context between implementation, profiling and research, and it lacks depth in each of them. After a few failures it picks the path of least resistance. Context doesn't last forever: findings from past sessions get lost, and hard ideas get filed as “dead ends” and never revisited.
Strength. An LLM agent works well with a narrowly defined role and can read the files you hand it. It's also pretty good at diagnosing someone else's work if you give it a log and a list of typical failures. That's just reading and analyzing text.
How the method uses this. Diagnosis and execution are separated: the supervisor doesn't write code, it reads the traces of the work and issues new mandates. Shared memory lives in files, so experience isn't lost when agents restart. For the “dead ends,” a fresh-eyes role is introduced to revisit abandoned ideas.
Control levers: - List of failure patterns → add your own (e.g., “the agent rewrites the tests instead of the code”) and the diagnosis gets sharper. - Trigger (on a timer or when work finishes) → a timer catches plateaus earlier, but you spend tokens on checks. - Roles → give them concrete names and prohibitions (“The Debugger doesn't touch the main branch”); this splits the work more cleanly. - Required reading → make new teams read the summary document, otherwise findings get lost.
Prompt Template
The accessible part of the paper contains no verbatim prompts from the authors. Below is a template assembled from the method description: the axis structure, the diagnosis patterns and the file system are preserved.
<Role>
You are the supervisor of an agent team. You don't write code. You observe the team,
diagnose the causes of stagnation and restructure the team.
</Role>
<Task>
Team goal: {goal}
Metric: {metric}, currently {current_value}, target {target_value}
Correctness constraints: {checks_and_tests}
</Task>
<KnowledgeBase>
Read before every decision:
- {path}/learnings.md — observations confirmed by measurements
- {path}/artifacts/ — results and notes from every team and role
- {path}/evolution.md — your past diagnoses and decisions
</KnowledgeBase>
<Trigger>
Run when: {current_team_finished_work} OR {every_N_minutes}
</Trigger>
<DiagnosisLogic>
Check progress and find the pattern:
- Drift toward the easy path: hard directions dropped after the first failures
- Loss of insights: findings from previous teams aren't being used
- Stuck in debugging: one problem blocks the rest of the work
- Micro-optimization: big effort, small gain
- {your_pattern}
</DiagnosisLogic>
<RestructureLogic>
Design team v+1 along three axes.
Roles:
Invent roles for the diagnosis; don't take them from a ready-made list.
For each: name, mandate, what it does NOT do, what it outputs.
Examples: Profiler, Skeptic (checks the “ceilings”),
Technique Researcher, Knowledge Integrator, Fresh Eyes (revisits
“dead ends”), Workload Analyst.
Coordination:
Choose: pipeline (research → implementation → measurement) /
parallel branches / both.
An agent debugging a blocking problem doesn't touch the main branch.
KnowledgeManagement:
For each role: where to write results, what to read before starting.
If insights are getting lost, introduce a summary document and make reading it mandatory.
</RestructureLogic>
<Output>
1. Diagnosis (grounded in logs and measurements)
2. Team lineup and each role's mandate
3. Coordination scheme
4. Knowledge-recording rules
5. Entry for evolution.md: observation → diagnosis → change → expectation
</Output>
What to fill in: {goal}, {metric} and the values: a measurable result; without a metric the supervisor has nothing to diagnose against. {checks_and_tests}: what must not break. {path}: the folder with the knowledge files. {your_pattern}: an agent mistake typical for your task.
🚀 Quick start: paste into the chat:
Here is the Evolving Agent Teams template. Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about a measurable metric, correctness constraints, the folder structure and typical agent mistakes in your task. Plateau diagnosis only works with a metric and a correctness check, and the failure patterns need to be tied to your work. The model will take everything else from the template.
Limitations
⚠️ Cost: The method was tested on multi-day runs with a big budget. For a half-hour task, this kind of scaffolding won't pay off.
⚠️ Needs a metric and a correctness check: The diagnosis relies on numbers and tests. For subjective tasks (“make it prettier”), the supervisor has nothing to grab onto.
⚠️ Weak statistical evidence: Each experiment was run twice and the best result was taken, with a single model family (Claude). The authors themselves admit this limits the conclusions.
⚠️ Not always a breakthrough: On one of the tasks, under a controlled budget, the team arrived at the same solution as the single agent. The best solution only appeared with a larger budget. On some workloads the result stayed below the expert baseline.
⚠️ No prompts provided: The accessible part of the paper has no verbatim supervisor instructions. The template above is a reconstruction from the description, and you'll have to tailor the roles yourself.
⚠️ Orchestration: The full version launches agents on a timer and maintains the knowledge folder. In Claude Code this can be set up, but it takes manual work.
How It Was Studied
The authors took four hard GPU-kernel acceleration tasks: Flash Attention 4, Gated DeltaNet, MLA decode and fusing GEMM with AllReduce. The reference points were expert implementations (FlashInfer, CUTLASS and others). They compared against plain Claude Code with the same token budget. Separately, they tested a fixed team of four “sensible” roles and Codex.
The surprise: even strong single agents hit a ceiling. Codex reached only 28% of the expert with a budget of about $1K. The fixed four-role team got to 55%. That means the issue isn't model strength or the number of agents, but a mismatch between the team's organization and the current phase of work. Evolving Agent Teams went through more than 100 team configurations on MLA. At an equal budget it hit 0.74× the expert versus 0.51× for Claude Code. With an extended budget it beat the expert (1.21× on average, faster on 36 of 47 workloads).
On another task, GDN, the extended budget pushed the speedup from 2.2× to 4.06×. The extra time helped precisely the hard workloads where the first architecture was a poorer fit. The practical takeaway: rebuilding the team around a diagnosis pays off when a task has several phases and each one calls for different skills.
Adaptations and Extrapolations
🔧 Technique: add a “Fresh Eyes” role → revisit the “dead ends”
The authors introduced a role that re-examines directions abandoned after failures. Add a line to <RestructureLogic>:
If learnings.md has “didn't work” entries older than {N} versions,
create a “Fresh Eyes” role: it re-examines every dead end
and decides whether the failure was caused by the idea or only by the implementation.
🔧 Technique: ban the main branch → debugging doesn't block everyone else
For a blocking problem, launch a separate agent with the rule: “Work in the debug/{name} branch. Don't commit to main. Deliverable: a note in artifacts.” A parallel agent keeps optimizing main.
Extrapolation: a lightweight version for a single agent in CLAUDE.md
If a full supervisor is overkill, add a self-diagnosis rule to the agent's instructions:
Every {N} iterations, stop and check yourself against this list:
1. Did the metric improve by less than 2% over the last iterations?
2. Did I drop a hard idea after the first failures?
3. Am I using the findings recorded in /knowledge/learnings.md?
If the answer is “yes” to at least one item, write the reason in evolution.md
and change direction or go back to the abandoned idea.
This isn't the authors' method, but a simplification of their diagnosis patterns for a single agent.
Resources
- Evolving Agent Teams — Shiyi Cao, Ziming Mao, Dacheng Li, Joseph E. Gonzalez, Ion Stoica, UC Berkeley
- Code: https://github.com/caoshiyi/EAT
- Related work cited in the paper: FlashInfer and FlashInfer-Bench, K-Search, Meta-Harness, AlphaEvolve, ADAS, Reflexion