HMA (Humanize MLE Agents): two agents take turns with a clean context and a shared folder

STEP 1: Shared working folder (code, results, best candidates) → persists between sessions
STEP 2: Agent A runs ≤ k experiments (k ≤ 5) → the session is cut off
STEP 3: Context is reset → agent B opens the folder, continues for ≤ k experiments
STEP 4: Repeat A ↔ B until time runs out (reserve the last ~4% of the time)
STEP 5: Final review: an agent that did not write the latest solution picks the best candidate from the archive

Each session is a separate agent run. The handoff happens only through files, not through conversation memory.

Task: an Amazon seller wants a better weekly sales forecast for 300 SKUs. They have two years of sales data, a baseline model script, and a metric: mean absolute error (MAE) on the held-out last 8 weeks. They have two subscriptions: Claude Code and Codex CLI. Instead of one marathon session, they run the two in turns in the same folder.

The paper has no ready-made prompts: the authors deliberately added no instructions. Below is my implementation of the mechanics, not a quote from the paper.

Prompt (identical for both agents, every session is a brand-new chat):

<Role>
You are an analyst improving the weekly sales forecast for an Amazon seller.
You work in tandem with another AI agent: you take turns refining the same folder.
</Role>

<Task>
Goal: reduce MAE on the held-out last 8 weeks (file data/sales.csv).
Baseline model: baseline.py. The metric is computed by the evaluate.py script.
Use training data only from before the held-out period.
</Task>

<SessionRules>
1. Start by reading WORKLOG.md and the experiments/ folder. That is your entire memory.
2. Run no more than 5 experiments. Experiment = one change + a run of evaluate.py + a logged result.
3. After each experiment, add a line to WORKLOG.md: what you changed, MAE, conclusion.
4. If the result beats the best so far, save it as best/ and note it in WORKLOG.md.
5. After the 5th experiment, write a HANDOFF block in WORKLOG.md: what worked, what didn't, which ideas haven't been tried. Then stop.
</SessionRules>

Final review prompt (for the agent that did not run the last session):

You are a reviewer. Read WORKLOG.md and experiments/. Pick the single best candidate
to submit. Check that there is no data leakage from the held-out 8 weeks and that the result
reproduces when you run evaluate.py. If the best candidate per the log fails the
check, pick the next one. At the end, state your choice and the reason.

Result: each session leaves up to 5 entries in WORKLOG.md plus a handoff block for the next agent. Starting from a clean context, the agent reads the log and picks up ideas its partner hasn't tried. After 6–10 handoffs, the best candidate sits in best/. The final agent confirms the pick or replaces it if it finds data leakage or a result that doesn't reproduce.

The weakness. In a long session, context piles up: old dead ends, long logs, early hypotheses. According to the authors, the agent's responses shrink noticeably by hours 3–5, and fresh ideas dry up. On top of that, a single agent hits the ceiling of its own habits. The one who found a good candidate isn't necessarily the best at polishing it.

The strength. An agent with a clean context comes back to the task fresh and "thinks" just as much in its first responses as at the very start. Files on disk hold state better than any context. Different models take different paths: one adds features, another fixes the validation. Handing over the result as a new starting point reveals what the first agent couldn't see. That's the authors' own explanation, not a separately tested fact.

Control knobs: - Session size k: the authors tuned it empirically, k ≤ 5. The first 5 experiments of a session deliver roughly 98% of all observed gains. Go lower for short tasks; for expensive experiments you can also go lower to pass the baton more often. - Agent pairs: different providers work best. Pairs of mismatched strength also improved results, and the Claude↔GPT combo won regardless of who started first. - Review at the end: a time reserve (15 minutes out of 6 hours in the paper). The reviewer must not be the author of the latest solution. - Native harness: don't pile your own mega-instructions on the agent. First check that it actually does worse without them.

<Role>
You are a {specialist_role} improving {what_we_improve}.
You work in tandem with another AI agent: you take turns refining the same folder.
</Role>

<Task>
Goal: {measurable_goal_and_metric}.
How to check the result: {command_or_evaluation_method}.
Constraints: {what_not_to_do}.
</Task>

<SessionRules>
1. Start by reading {log_file} and the {experiments_folder} folder. That is your entire memory.
2. Run no more than {k} experiments. Experiment = one change + a measurement + a logged result.
3. After each experiment, add to {log_file}: what you changed, the result, the conclusion.
4. If the result beats the best so far, save it to {best_folder}.
5. After the last experiment, write a HANDOFF block: what worked, what didn't, which ideas haven't been tried. Then stop.
</SessionRules>

What to fill in: - {k} — 5 by default. - {measurable_goal_and_metric} — a number and how to get it: MAE, conversion rate, run time. - {log_file} — one shared file that both agents read.

For the finale, run a separate reviewer (the other agent) with the prompt: "pick the best candidate from the log, check reproducibility and that there's no leakage."

🚀 Quick start — paste into a chat:

Here is the HMA template (two agents alternating with a shared folder and context resets). Adapt it to my task: [your task]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask how to measure success, what counts as one experiment, and where the shared state lives. Without a measurable metric and a shared log, the handoff between agents doesn't work. It will take the pattern from the template and adapt it to your task.

⚠️ You need a measurable metric: the method was tested on tasks where an experiment's outcome is a number (ML competitions, code optimization). For writing, design, and strategy, where "better" can't be measured, the paper offers no evidence.

⚠️ You need two agents from different providers: that means two subscriptions or two API keys, plus manual or automated handoff between tools. In the paper, orchestration was handled by the ready-made Humanize framework (code available). The authors did not test manual alternation.

⚠️ The gain is moderate: the improvement over the best solo agent is a few percentage points. The final review corrected only about 9% of picks and added a handful of medals out of 450 runs.

⚠️ One task type and one hardware setup: the main results come from 75 Kaggle competitions with a GPU and a six-hour budget. The value k ≤ 5 was tuned on that same set. The kernel optimization case is a single task.

⚠️ Harness aging was measured narrowly: "harnesses go stale" was shown on one pair of model versions and 16 tasks. It's a strong signal, not a law.

⚠️ The slowdown was measured indirectly: "reasoning intensity" is response length. It declines along with the gains, but causality isn't proven.

⚠️ Some ablations aren't visible: the provided text is truncated, so the breakdown of individual method components (what the context reset itself contributes versus switching agents) can't be checked here.

Researchers from Tsinghua, NVIDIA, MIT, and Zhejiang University used MLE-bench: 75 Kaggle competitions where an agent writes and runs ML solutions. Each agent got one A10 GPU and six hours. First they looked at what happens to solo agents. Across six configurations, response length dropped by 29.8–45.0% between hours 0–2 and hours 3–5. Then they compared three third-party custom harnesses (ScienceFlow, ML-Master 2.0, MLEvolve) on two versions of the same model. On the weaker version, the custom approaches sometimes beat the native harness. On the new version, the native harness earned 77.1% medals versus 37.5% to 68.8% for the custom ones, even though those were given 12–24 hours.

Then they assembled HMA: 5 model pairs, 6 configurations, k = 5, the final 15 minutes for review. The best pair (Opus 5 ↔ GPT-5.6-sol) earned 78.2% medals versus 72.4% for the best solo agent and 70.2% for the best open system. With the starting order reversed it got 73.3%, still above 72.4%. There was a gain in every one of the six configurations, averaging +6.4 points over the component agents. One more result: HMA retains 86.4% of its early-stage response length in the late stage, while solo agents retain 60.7%. The practical takeaway: rather than making the harness more elaborate, it pays to reset context often and bring in "fresh eyes."

🔧 Technique: revisit your instructions when the model changes → less dead weight

The authors cite vendor recommendations: OpenAI advises rewriting AGENTS.md, skills, and prompts when upgrading the model; Anthropic advises removing instructions for checks that newer models do on their own. The practical move: after upgrading your agent, run a task without your CLAUDE.md and compare it to the version with it. If there's no difference, trim the file.

Extrapolation: a "clean chat" in everyday work. This is my own loose transfer of the idea, not a result from the paper.

We've been working on {task} for {N} iterations. Stop. Write me:
1. The current best version (in full).
2. What we tried that didn't work.
3. Three ideas we haven't tried yet.
I'll open a new chat and paste this in as the starting state.

Then open a new chat (ideally with a different model) and paste in this block along with the task. Same principle: state lives in text, not in a long conversation. The authors did not measure the effect in a regular chat.

• Paper: Humanize MLE Agents: Multi-Agent Session Alternation Beyond Single-Agent Limits (Preprint)
• Code: https://github.com/humanfia/hma
• Authors: Changye Li, Zijian Zhang, Muheng Li, Yiduo Pang, Tingjun Wu, Menghan Li, Yitong Liu, Hongkai Guo, Mengchi Chen, Chenyu He, Kaisheng Ma, Wei Xu, Song Han, Sihao Liu, Ligeng Zhu
• Organizations: Tsinghua University, NVIDIA, Zhejiang University, MIT
• Benchmark: MLE-bench (Chan et al., 2025)