How many examples does few-shot prompting actually need

The question of how many examples few-shot prompting actually needs has a frustrating answer: it depends on the task, the model, and whether you’re optimizing for latency, cost, or quality. But after running thousands of evaluations across model families and task types, a clear pattern emerges — most production workloads plateau between 3 and 8 examples, with diminishing returns setting in hard after 10. The real lever isn’t example count; it’s example quality and diversity.

The diminishing returns curve is real

Every model family exhibits a similar curve. Zero-shot works surprisingly well for classification and extraction tasks where the label space is obvious. One-shot adds the output format constraint. Two to three shots teach the pattern. Beyond that, you’re paying token costs for marginal gains that often disappear under evaluation noise.

I’ve seen this across classification, entity extraction, code generation, and structured output tasks. A typical curve looks like this:

The jump from 0 to 3 shots is where the value lives. The jump from 5 to 12 is noise.

Why the curve flattens

Three mechanisms drive the plateau:

Context window saturation. Each example consumes tokens that could go to the actual input. For a 4k context model with 500-token examples, 8 shots leaves 1k tokens for the real work. That’s a real constraint on long-document tasks.

Pattern saturation. Models learn the output format and label distribution within 3-5 diverse examples. Additional examples of the same pattern add nothing. They only help if they cover genuinely distinct cases — edge cases, rare labels, ambiguous boundaries.

In-context learning mechanics. The attention mechanism has finite capacity to “bind” examples to the query. Beyond a certain density, examples interfere with each other rather than reinforce. This is why random ordering of few-shot examples sometimes outperforms curated ordering — it prevents the model from overfitting to positional artifacts.

Task-type breakdown

The optimal shot count varies systematically by task category.

Classification and labeling

Sweet spot: 3-5 shots per class. For binary classification, 3-5 total examples often suffices. For multi-class with 10+ labels, you need at least 2-3 per class, which drives the total up. But even with 20 classes, 40-60 examples is usually the ceiling — beyond that, fine-tuning or RAG beats in-context learning.

# Typical few-shot classification prompt structure
examples = [
    {"text": "The battery life is amazing", "label": "positive"},
    {"text": "Screen cracked after two days", "label": "negative"},
    {"text": "It's okay, nothing special", "label": "neutral"},
    # ... 2-3 more per class
]

Key insight: balance matters more than count. An imbalanced 10-shot prompt (8 positive, 2 negative) performs worse than a balanced 6-shot prompt.

Entity extraction and NER

Sweet spot: 5-8 shots covering entity types and boundary cases. Extraction tasks need examples that demonstrate:

• Each entity type at least twice
• Nested entities if applicable
• Boundary ambiguity (e.g., “New York City” vs “New York”)
• Negative examples (sentences with no entities)

{
  "text": "Apple announced the iPhone 15 in Cupertino yesterday.",
  "entities": [
    {"start": 0, "end": 5, "type": "ORG"},
    {"start": 22, "end": 32, "type": "PRODUCT"},
    {"start": 36, "end": 45, "type": "LOC"}
  ]
}

Eight well-chosen examples beat twenty generic ones.

Code generation

Sweet spot: 3-6 shots with varying complexity. Code tasks benefit from examples that show:

• Simple happy path
• Error handling pattern
• API usage idioms
• Test case structure

The model learns the style and conventions quickly. Additional examples mainly help with library-specific patterns.

Structured output (JSON, SQL, regex)

Sweet spot: 4-8 shots. Format adherence improves sharply with 3-4 examples showing the exact schema. Beyond that, gains come from covering optional fields, nested structures, and constraint violations.

Quality beats quantity: the example selection checklist

If you’re debating between 5 and 8 examples, you’re asking the wrong question. Ask instead:

Do examples cover the label distribution? If your production data is 80% class A, 15% class B, 5% class C, your few-shot set should roughly mirror this — or deliberately oversample rare classes if recall on those matters.

Do examples cover failure modes? Include the cases where your zero-shot prompt fails. If the model confuses “billing issue” with “refund request,” add examples of both.

Are examples diverse in surface form? Five examples that differ only in entity values teach nothing. Vary sentence structure, vocabulary, negation, hedging.

Are labels correct? This sounds obvious. But I’ve seen production prompts with mislabeled examples that the model faithfully learned to replicate. Audit your few-shot set.

Is the format consistent? Every example must use the exact output format you want. Mixed formats (some JSON, some markdown) confuse the model.

The hidden cost: token budget and latency

Each example adds tokens to every request. At scale, this compounds.

Input tokens per request = system_prompt + few_shot_examples + user_input

For a classification task with 500-token examples:

• 3 shots: +1,500 tokens/request
• 8 shots: +4,000 tokens/request
• 12 shots: +6,000 tokens/request

At 10k requests/day on a model charging $0.50/M input tokens:

• 3 shots: $2.25/day
• 8 shots: $6.00/day
• 12 shots: $9.00/day

Latency scales similarly. More context = longer prefill = higher TTFT. For latency-sensitive paths, this matters.

When more shots actually help

There are legitimate cases for 10+ shots:

Many-class classification with long-tail distribution. If you have 50 classes and the bottom 30 appear rarely but matter, you need examples for each. Consider switching to a retrieval-based approach (RAG over labeled examples) instead of stuffing all into context.

Complex multi-step reasoning. Tasks requiring chain-of-thought with specific reasoning patterns (e.g., medical diagnosis, legal analysis) benefit from more demonstrations of the reasoning structure.

Style transfer and tone matching. If the task is “write in the voice of X,” more examples capture more stylistic range.

Adversarial robustness. If users actively try to break your prompt, more diverse examples harden the boundary.

In these cases, consider dynamic few-shot: retrieve the k most relevant examples for each query rather than using a fixed set. This keeps token count bounded while scaling example diversity.

Evaluation protocol that prevents overfitting

You cannot trust a single evaluation run. Few-shot performance has high variance across example ordering, example selection, and random seeds.

Run this protocol:

def evaluate_few_shot(task, example_pool, shot_counts, n_trials=10):
    results = {}
    for k in shot_counts:
        scores = []
        for trial in range(n_trials):
            examples = random.sample(example_pool, k)
            prompt = build_prompt(task, examples)
            score = run_evaluation(prompt, task.test_set)
            scores.append(score)
        results[k] = {
            "mean": np.mean(scores),
            "std": np.std(scores),
            "ci95": 1.96 * np.std(scores) / np.sqrt(n_trials)
        }
    return results

Key practices:

• Sample examples randomly for each trial (not fixed)
• Use at least 10 trials per shot count
• Report confidence intervals, not point estimates
• Test on held-out data the model hasn’t seen during prompt development

If the 95% CI for 5 shots overlaps with 8 shots, stop at 5.

The routing consideration

When you’re running multiple models behind a single endpoint, few-shot sensitivity varies by model. Smaller models (7B-13B) often need more examples to reach the same performance as larger models (70B+), but they also have smaller context windows. This creates a tension: the models that need more shots have less room for them.

One practical approach: maintain per-model few-shot configurations. A 7B model might get 8 carefully selected examples; a 70B model gets 3. The routing layer selects the model and applies the appropriate prompt template. This is where a gateway that honors client routing directives and forwards provider cache-control hints becomes useful — you can optimize per-model without fragmenting your application logic.

Decision framework

Use this flowchart when someone asks “how many examples?”

1. Is it zero-shot viable? Test zero-shot first. If F1 > 0.85, stop.
2. What’s the task type? Classification → 3-5. Extraction → 5-8. Code → 3-6. Structured output → 4-8.
3. How many classes/labels? Multiply by 2-3 per class. Cap at 20-30 total for in-context.
4. What’s the token budget? Calculate max shots = (context_window - system_prompt - max_input - reserved_output) / avg_example_tokens.
5. Run the evaluation protocol. Find the elbow. Stop there.
6. Monitor in production. Track performance by example subset. Remove examples that correlate with errors.

The decisive takeaway

Stop optimizing example count. Start optimizing example quality.

Three diverse, correct, failure-mode-covering examples beat ten redundant ones. The marginal gain from shot 4 to shot 10 is almost always smaller than the gain from replacing two mediocre examples with two great ones. Your time is better spent auditing your few-shot set for label errors, coverage gaps, and format consistency than debating whether 6 or 7 is the magic number.

If you need more than 10 examples to get acceptable performance, the problem isn’t example count — it’s that in-context learning is the wrong tool. Switch to fine-tuning, RAG, or a specialist model.