Skill Routing Collapse: the more skills you install, the less often the agent opens the right one

STEP 1 (separate request, not in the agent's working chat):
  task + “name + description” list of all skills
  → pick up to 5 needed → list of names

STEP 2 (working request to the agent):
  task + only the selected skills
  → the agent opens them and gets to work

For libraries with thousands of skills (hybrid):
  the LLM writes several search queries for the task
  → candidate search for each query
  → the LLM picks the final ≤5 from the combined pool

Steps 1 and 2 are two separate requests. Don't merge them into one: that's exactly the anti-pattern.

Task: An Amazon seller has built a library of ~150 skills in Claude Code: he downloaded a batch from a skills marketplace and added a few of his own. He needs to take a sales export from Seller Central, calculate unit economics per SKU, and build a deck for a supplier. He notices the agent often does everything “from scratch” and ignores his Amazon fee-calculation skill.

Prompt (step 1, in a separate chat):

You are selecting skills for an agent. Below are a task and a list of skills
(name: description). Pick no more than 5 skills that
will actually be needed for this task. Don't solve anything yourself,
just pick.

Task: from an Amazon sales export (xlsx, 40,000 rows)
calculate unit economics per SKU, accounting for Amazon fees,
fulfillment and returns, build 3 charts and put together
an 8-slide deck for the supplier.

Skills:
{paste the “name: description” list from the skills folder}

Response format: a list of skill names, each with one line
on why it's needed.

Prompt (step 2, in the working agent):

Task: from the file sales_amazon.xlsx, calculate unit economics per
SKU (Amazon fees, fulfillment, returns), build 3 charts
and put together an 8-slide deck for the supplier.

You must open and use these skills: amazon-fee-calc,
xlsx, charts, pptx. If you think another skill is needed, first
say which one and why.

Result: The first chat returns a short list of a few skills with explanations. In the second, the agent should open exactly those instead of trying to write the calculations and slides on its own. You can verify this in the agent's work log.

The weakness. While solving a task, the agent is also juggling a long list of descriptions. Descriptions of unrelated skills read like plausible alternative actions, and that mush drowns out the right ones. Choosing a skill also competes with the work itself: the agent has already started reading files and writing commands and doesn't stop to pick skills. The authors showed this two ways. Hide only the descriptions of the extra skills, and loading nearly returns to baseline. Just ask it to “think about skills first,” and the gain is small, far behind a separate request.

The strength. When picking skills is the request's only goal, the model does well. Even with a thousand skills it finds most of the right ones. The right skills are recognizable; you just can't combine choosing with doing.

How the method exploits this. Preselection turns choosing into a separate, clean task. The agent gets a short list and doesn't burn attention on choosing.

Levers: - Number of skills in the output list (≤5 in the study). Fewer means less noise but a higher risk of missing one you need. - Selection method. Up to roughly a hundred skills, a simple LLM request is enough. At a thousand and beyond, a hybrid works better (several queries + search + LLM). Plain similarity search works too, but less precisely. - List size in the request. A request with the full list of 10,000 skills simply doesn't fit in the context. - Naming the skill explicitly in the task. This is manual preselection, the cheapest option.

You are a skill dispatcher. Your only job is to choose which
skills are needed for the work. Do not do the work itself.

<Task>
{task_description}
</Task>

<Skills>
{name_1}: {description_1}
{name_2}: {description_2}
...
</Skills>

Rules:
- pick no more than {skill_count} skills;
- pick only the ones without which the task gets done worse;
- don't pick a skill “just in case”;
- if none fit, say so.

Response format: a list of “name — why it's needed (one line)”.

What to fill in: - {task_description} — the task text exactly as you'll hand it to the agent. - The <Skills> block — “name: description” lines from the skill folders (in SKILL.md these are the name and description fields). - {skill_count} — 5 in the study. For your task, start with 3–5.

Then, in the working request to the agent, list the chosen skills by name and ask it to open them before it starts.

⚠️ Preselection can cut out what you need: skills chosen up front won't include what turns out to be needed later in the work. In the study, the share of needed skills available to the agent is noticeably lower with preselection than with the full list.

⚠️ It's a mitigation, not a cure: even the best option didn't bring skill loading back to the “no extra skills” level, especially on very large libraries.

⚠️ Latency: preselection adds a few seconds of waiting before the start, more for the hybrid version.

⚠️ Experiment scope: one task suite (SkillsBench), with the extra skills drawn at random from a public marketplace. With your own skills, where descriptions resemble each other, the effect may differ, most likely it will be stronger.

⚠️ No universal numbers: the main experiments ran on open models; closed ones (Claude, Gemini, GPT) were tested on a smaller sample and showed the same trend. Don't carry the numbers over to your case, carry over the direction.

⚠️ Small libraries: with a dozen skills the drop is small. It becomes noticeable at around a hundred.

⚠️ The paper is truncated: the explanation of exactly how descriptions play “two roles” is cut off in the provided text; the conclusions below rely on the parts given in full.

The researchers took the existing SkillsBench benchmark: 82 tasks with verifiable outcomes (a code verifier, no LLM judge) and “gold” skills known in advance. Then they changed one variable: how many extra skills were added to the library (0, 10, 100, 1,000, 10,000). The task, the files and the needed skills stayed the same. Extra skills were drawn at random from 20,000+ skills on a real marketplace, and three open models were each run three times: over 15,000 runs and 10 billion tokens. Closed models were tested on a subset.

The main metric is the share of needed skills the agent actually opened, plus the share of runs where none were opened. Control question: maybe the agent swaps in other, similar skills? No: that's rare (about 2% of runs), and these are essentially duplicates like “pdf-reader” instead of “pdf”. The collapse is under-loading.

Then the authors tested the cause. In the first experiment, they hid the names and descriptions of the extra skills behind placeholders of the same length. The context stayed long, but loading almost recovered. So the culprit is the meaning of the descriptions, not the length. In the second, they asked the agent to pick skills separately from solving. The choice came out far more accurate than when the agent was asked to “plan the skills and solve.”

The surprise: overall task success drops far less than skill loading, because a strong agent can solve the task without the skill. Practical takeaway: don't look only at the result, check whether the skill was opened.

💡 Adaptation for manual work (the simplest form of preselection): instead of a separate request, just name the skills in the task.

Copy

Use these skills: {skill_1}, {skill_2}. Open them before you start.
Task: {task}

This is manual preselection that you do yourself. With a library of a hundred skills or more, it's the cheapest way to restore loading. The paper didn't test this variant directly, but it follows from the finding that separate selection works better.

🔧 Technique: load check (extrapolation, not tested in the paper)

Add this to the end of the task: “At the end of your answer, list which skills you opened and which you deliberately skipped.” That way you'll quickly notice when the agent worked without skills, even if the result looks fine. The paper shows you can't tell from the final result alone.

🔧 Technique: library cleanup (extrapolation, not tested in the paper)

The paper shows that it's specifically the descriptions of extra skills that create the noise. So: disable skills you don't need in a given project, and write descriptions that are narrow and distinct from one another. To be fair: only stripping the meaning out of descriptions entirely was tested, not rewriting them.

• Lost in Implicit Routing: How LLM Agents Fail to Load Relevant Skills at Scale — Jinlin Liu (The University of Hong Kong), Wei Chen and Xiaojin Zhang (Huazhong University of Science and Technology)
• Code and data: github.com/jinlinll/lost-in-implicit-routing
• SkillsBench (Li et al., 2026) — the benchmark the experiments ran on
• Anthropic Agent Skills — the skill format with progressive disclosure that was studied