Few-shot prompting: what the research shows

What few-shot prompting is

Few-shot prompting puts a handful of worked examples into the prompt so the model can copy the pattern instead of inferring it from a description. An example can show only input and output, or input, reasoning and output. The second form, few-shot chain-of-thought, is what a study on reading hidden intentions used to bring GPT-4o to human level on inferring what a vague request really meant, where zero-shot and plain answer-only examples fell short.

In practice the examples rarely travel alone. The prompts that tested well combine three parts: a definition of the task or output schema, the examples, and the new input. A clinical extraction study pairs a JSON schema with one or two reference answers; a fact-versus-opinion classification study pairs detailed category definitions and edge cases with 2-6 examples, and that plain setup beat multi-agent debates.

Why examples work, and how they mislead

Examples work because models are strong pattern matchers within the context window. They show the format, the level of detail and the boundary between categories in a way a paragraph of instructions often cannot. Two consequences follow from the same mechanism.

• The model copies form, not intent. In Show and Tell, 160 coding sessions with Gemini 2.5 Pro showed that concise examples produced concise first drafts, but once the user asked to "improve the code" the output bloated exactly as if no examples had been given. Explicit rules kept working on revision; rules plus examples cut first-turn tokens by 70%.
• The model cannot tell a good example from a bad one. In a GUI agent that reuses its own past solutions, switching off the quality filter made the agent worse than having no example library at all; only trajectories scored at least 4 out of 5 were safe to reuse.

Examples also shift what the model thinks the task is. In a jailbreak-defense study across 9 models, safe-refusal examples strengthened a prompt framed as a role ("you are a safe assistant", up to +4.5%) but weakened a prompt framed as a list of rules (down to -21.2%), because the extra example tokens drew attention away from the instructions.

How many examples, where, and which ones

The studies give fairly concrete settings.

• How many. On 90,000 code translations with Gemini models, accuracy peaked at 61-62% with 5-25 examples and fell back to 54-58% at 125-625, where the model copied surface patterns; the large prompt also cost 21 times more (the many-shot paradox).
• Where. Examples placed before the question, ideally at the start of the system prompt, beat examples appended after it by up to +6% and flipped fewer answers (positional bias study). For code, putting test cases inside the function's docstring rather than in a block at the end raised correct outputs to 92–100%, because the link between example and task was 2.8–4.4 times stronger (co-located tests).
• Which ones. Cover the distinct cases you care about. One prompt with 6 examples of different meaning shifts (number, polarity, obligation, dropped condition) beat an 8-agent pipeline at spotting translation errors in technical requirements and ran in 17 minutes instead of 181 (Direct Judge). Keep examples consistent in tone; mixed styles confuse the pattern.
• Rare formats. When the input looks unlike training data, two examples can restore performance: on numbers written in unusual scripts and separators, accuracy that had dropped by 87% climbed from 13% to 70-90% (multiscript numeracy).
• Your own history. Past question-answer-result cases worked better than a self-description for personal advice (track record versus bio), and compressing an old case into a short note of what to repeat and what to re-check stops the model from copying stale names and dates (query-conditioned reuse).

A related option is to let the model write its own example first; that is covered on the analogical prompting page.

When rules or reasoning beat examples

The evidence is not one-sided, and the studies disagree on how much examples add once instructions are good.

• Rules beat examples for compound logic. Across five invented tasks, an explicit rule ("URGENT if A OR B, AND C") beat a set of unexplained examples, and adding examples to a clear rule barely helped (rules versus examples). The gap closed on tasks that rely on categories the model already knows.
• Forced reasoning can beat examples. For coding legal texts into categories, a step-by-step decomposition prompt outperformed the few-shot version (deductive coding study).
• But definitions plus examples beat fancier methods. The opinion-detection study above found even randomly chosen examples on top of detailed definitions were enough to beat debates between models.

A reasonable reading of all three: write the rule or definition first, then add a few examples to show format and edge cases. Expect examples to carry the first answer but not follow-up turns, use the role framing rather than a rule list if examples go into a safety prompt, and stop adding examples well before the hundreds.

You are a {role}. Here are {5-25} examples of {content type} that work well:
[Example 1: {context}]
{example text}
[Example 2: {different context}]
{example text}
...
[Example {N}: {another context}]
{example text}
---
Now create {content type} for a new context:
- {key information 1}
- {key information 2}
- {key information 3}
Keep {what matters to preserve from the examples}, but adapt it to {new context}.

def {function_name}({params}):
    """{One line: what the function does}
    >>> {function_name}({typical_input_1})
    {expected_result_1}
    >>> {function_name}({typical_input_2})
    {expected_result_2}
    >>> {function_name}({edge_case})
    {edge_case_result}
    """
    pass
{Additional requirements if any}
Write the implementation.

Compare two texts — the original and the {translation/paraphrase/version}.
Determine whether the meaning changed in the aspects that matter to me: {what matters — numbers, obligation level, conditions, objects}.
Here are examples of what counts as a divergence and what doesn't:
1. NO divergence: [example of a paraphrase with no meaning lost]
2. DIVERGENCE ({type 1}): [example]
3. DIVERGENCE ({type 2}): [example]
4. DIVERGENCE ({type 3}): [example]
5. DIVERGENCE ({type 4}): [example]
6. DIVERGENCE ({type 5}): [example]
Now compare:
Text A: {original}
Text B: {version to check}
Answer:
1) Divergence: YES/NO
2) Confidence: low/medium/high
3) What changed (if anything)