The Many-Shot Paradox: When More Examples = Worse Results

ZERO-SHOT (0 examples)
→ Baseline: ~54-57% success

FEW-SHOT (5-25 examples)  
→ PEAK performance: ~61-62% success
→ The model grasps the essence of the task

MANY-SHOT (125-625 examples)
→ DEGRADATION: back to ~54-58% success
→ The model copies surface patterns
→ Functional errors go up

Tipping point: after 25 examples, every additional 100 examples lower accuracy by 1-3%, even though the text may look better.

Task: You're a marketer at an online retailer. You write Amazon-style product listings — persuasive and emotional, but precise on the specs. You need to describe a new collection of winter jackets (50 models).

⚠️ DON'T do this: Copy 100 ready-made descriptions from the catalog → "Here are examples, now describe the new jackets in the same style"

✅ Do this:

Prompt:

You are a copywriter for Amazon product listings. Here are 8 descriptions of winter jackets from our catalog that sell well:

[Example 1: Alaska down jacket — focus on cold resistance]
Handles an Arctic -40°C, no joke. 90/10 down, windproof, extended cut — you stay warm even if you left your hat at home...

[Example 2: Urban parka — focus on style + practicality]
Looks like a million bucks, costs like... well, a lot less. 10K membrane, removable fur, power bank pockets...

[Examples 3-8 with different USPs: lightness, packability, durability, etc.]

---

Now describe the new model:
- Name: Alaska Luxe
- Features: 95/5 goose down, 15K waterproofing, carbon elbow panels, -50°C
- Target audience: men 30-45, active lifestyle, willing to pay for quality

Keep the Amazon style (emotion + precision), but adapt it to this model's specs.

Result: The model produces a description that keeps the style's DNA (emotion, concrete numbers, conversational phrasing), but accurately reflects the new model's USP. Give it 100 examples and the model starts mechanically mixing phrases from different listings, losing focus on what makes the product unique.

LLM weakness: The model doesn't "understand" the task — it looks for patterns in the examples. With 5-25 examples the pattern is clear and concentrated. With 100+ examples the patterns clash — some examples say "write short," others "write long," others "be serious," still others "be emotional."

LLM strength: Models generalize great from a small set of high-quality examples. 5-8 examples are enough to extract the essence of a style, tone and structure. It's like explaining a task to a person — 3-5 good examples beat 100 mediocre ones.

The paradox mechanics: Lots of examples = surface-level copying instead of deep understanding. The model starts imitating the form (sentence length, word frequency, syntax) but loses the substance (logic, correctness, uniqueness). The result: text that looks plausible but performs poorly.

Control levers:

• Number of examples (5-25 is the sweet spot) → fewer = saves tokens but may miss the style; more = noise and degradation
• Example quality → pick diverseexamples, all within one pattern (different USPs, same tone)
• Explicit instruction ("keep the style, but adapt it to the new context") → helps the model avoid blind copying

You are a {role}. Here are {5-25} examples of {content type} that work well:

[Example 1: {context}]
{example text}

[Example 2: {different context}]
{example text}

...

[Example {N}: {another context}]
{example text}

---

Now create {content type} for a new context:
- {key information 1}
- {key information 2}
- {key information 3}

Keep {what matters to preserve from the examples}, but adapt it to {new context}.

How to fill it in:

• {role} — who you are for the model (copywriter, analyst, translator)
• {5-25} — keep it in the 5-25 example range, no more
• {content type} — what you're creating (product description, customer email, data analysis)
• {context} for each example — show a variety of scenarios
• {what matters to preserve} — spell out what the model should take from the examples (style, structure, tone)

🚀 Quick start — paste into the chat:

Here is a few-shot template of the optimal size. Adapt it to my task: {your task}. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what examples you have, what result you need and what style to preserve — because it needs to understand what to extract from the examples and how to apply it to the new context. It will take the prompt structure and build a working version for your task.

⚠️ Simple tasks: For obvious tasks (translating a word, a country's capital), even 5 examples are overkill. Zero-shot works great.

⚠️ Examples that are too different: If your 10 examples come in different styles/tones/formats, the model gets confused. Better 5 examples in one style than 20 mismatched ones.

⚠️ Subjective criteria: If the task is about "creativity" or "originality," examples can limit the model's imagination. For creative work, go zero-shot or 2-3 examples "for inspiration."

⚠️ Tokens and money: 625 examples = 20-30x more expensive than 25 examples. If the result is worse and the price is higher, it's simply a bad deal.

The team took 90,000 code translations between 6 programming languages (C, C++, C#, Java, Go, Python) and ran them through 3 Gemini models with different numbers of examples: 0 (zero-shot), 5, 25, 125, 625. The key move: they measured not only "text similarity" (BLEU, CodeBLEU) but also functional correctness: does the code compile, does it run, does it produce the right result.

Here's what they found: static metrics (text similarity) grew or held steady as examples increased, but functionality dropped. With 25 examples, 61.8% of the code compiled successfully; with 625 examples, only 56-58% — almost zero-shot level. What's more, as examples grew, so did functional and runtime errors — the code looked right but worked wrong.

Why? With 625 examples the model learned to copy syntactic patterns (code structure, keywords) but lost semantic understanding (logic, correct behavior). It's like a student who memorized 100 worked solutions but never got the underlying principle — on a new problem they'll apply the wrong formula.

The economics: A prompt with 625 examples cost 21 times more than one with 25 examples (in tokens), yet gave a worse result. This isn't just an academic paradox — it's real lost money and quality.

Robustness of the finding: The paradox showed up in all 30 language pairs, including easy ones (Java → C#) and hard ones (Python → Go). Regardless of translation difficulty, the peak was always in the 5-25 example zone.

When Many-Shot Prompting Fails: An Empirical Study of LLM Code Translation — Amirkia Rafiei Oskooei, Kaan Baturalp Cosdan, Husamettin Isiktas, Mehmet S. Aktas (Yildiz Technical University, Intellica Business Intelligence, 2025)