Few-shot vs fine-tuning: which gets better results

The question of few-shot vs fine-tuning comes up on every team building with LLMs. Most engineers reach for few-shot first because it’s faster to iterate. But fine-tuning pays off when you need consistent behavior at scale, lower latency, or capabilities that prompting alone can’t unlock. This post breaks down the trade-offs across six dimensions so you can decide without running a month-long experiment.

Capabilities: what each approach actually buys you

Few-shot prompting relies on in-context learning. You stuff examples into the context window and hope the model generalizes. It works surprisingly well for classification, extraction, and style transfer — tasks where the pattern is visible in a handful of demonstrations. But it has hard limits: the model can’t learn new facts, can’t internalize complex multi-step reasoning patterns beyond what fits in context, and degrades when examples are noisy or contradictory.

Fine-tuning updates model weights. The model actually learns the task distribution. This means it can internalize reasoning patterns that would require dozens of few-shot examples, handle edge cases more consistently, and compress knowledge that would otherwise consume thousands of context tokens per request. A fine-tuned 7B model often outperforms a prompted 70B model on narrow tasks.

# few-shot: you pay for examples every request
messages = [
    {"role": "system", "content": "Classify support tickets."},
    {"role": "user", "content": "Ticket: \"Login broken\"\nLabel: auth"},
    {"role": "user", "content": "Ticket: \"Billing wrong amount\"\nLabel: billing"},
    # ... 20 more examples
    {"role": "user", "content": f"Ticket: \"{user_input}\"\nLabel:"},
]

# fine-tuned: zero examples at inference time
messages = [
    {"role": "system", "content": "Classify support tickets."},
    {"role": "user", "content": f"Ticket: \"{user_input}\"\nLabel:"},
]

The capability gap widens when you need structured output adherence, domain-specific terminology, or consistent tone across thousands of generations. Few-shot drifts. Fine-tuned models stay put.

Price and cost model

Few-shot looks cheaper upfront. You pay per-token at inference, and that’s it. But the per-request cost scales linearly with example count. Twenty examples at 200 tokens each adds 4,000 tokens per request — at $2.50/M input tokens, that’s $0.01 per classification. At 1M requests/month, you’re spending $10K just on context overhead.

Fine-tuning flips the model: high upfront cost, near-zero marginal cost. Training a 7B model on 10K examples might cost $50–$200 depending on provider and hardware. After that, inference is just the base model cost — no example overhead. The break-even typically lands between 500K and 2M requests for most tasks.

Hidden costs: few-shot requires prompt engineering time per task. Fine-tuning requires data curation, eval sets, and iteration cycles. Factor engineering hours — they dominate at small scale.

Latency and throughput

Every token in your few-shot prompt adds latency. A 4K-token prompt with 20 examples adds 50–150ms on typical inference endpoints before the model generates a single token. At high throughput, this consumes KV cache capacity and reduces batch efficiency.

Fine-tuned models run at base model latency. No prompt overhead. For latency-sensitive paths — autocomplete, real-time classification, chat — this matters. A fine-tuned 7B model at 50ms beats a prompted 70B at 300ms for most interactive use cases.

Throughput scales differently too. Few-shot prompts fragment the KV cache across unique example combinations, reducing prefix caching effectiveness. Fine-tuned requests share identical prefixes (just the system prompt), maximizing cache hit rates on providers that support it.

Ergonomics and iteration speed

Few-shot wins on iteration speed. Change an example, reload the prompt, test immediately. No GPU queue, no training run, no version management. This is why every team starts here.

# few-shot iteration: edit file, reload, test
vim prompts/classifier.txt
curl -X POST ... -d '{"prompt": "...updated examples..."}'

# fine-tuning iteration: curate data, train, eval, deploy
python prepare_data.py --input raw.jsonl --output train.jsonl
python train.py --model llama-3-8b --data train.jsonl --epochs 3
python eval.py --model ./checkpoints/step-1500 --test test.jsonl
# deploy new adapter, update routing

Fine-tuning introduces a deployment pipeline. You need eval harnesses, regression tests, rollback strategy, and model versioning. But once that pipeline exists, iterating on data is often faster than iterating on prompts — you’re teaching the model rather than coaxing it.

The ergonomics gap narrows with tooling. LoRA adapters train in minutes on consumer GPUs. Parameter-efficient fine-tuning (PEFT) means you can swap adapters without reloading base weights. Some platforms let you hot-swap adapters per request.

Ecosystem and tooling maturity

Few-shot works everywhere. Every provider, every open model, every framework supports it. No special infrastructure needed.

Fine-tuning ecosystem has matured rapidly but remains fragmented:

• OpenAI: GPT-3.5/4o fine-tuning API, managed but opaque
• Together, Fireworks, Anyscale: Managed LoRA/QLoRA on open models
• Unsloth, Axolotl, LLaMA-Factory: Local training frameworks
• vLLM, TGI, SGLang: Serving with multi-LoRA support
• Hugging Face Hub: Adapter sharing and versioning

The lock-in risk is real. OpenAI fine-tunes only run on OpenAI. Open model adapters run anywhere that supports the base model — but you own the serving infrastructure. n4n.ai handles this by routing fine-tuned adapters across providers that support the same base architecture, but the adapter format must match.

Limits and failure modes

Few-shot fails silently. The model follows the pattern until it doesn’t — edge cases, distribution shift, adversarial inputs. You discover failures in production. Context window limits cap example count. Token costs compound. Prompt injection risk increases with example surface area.

Fine-tuning fails loudly — during eval. Catastrophic forgetting, overfitting to small datasets, capability degradation on unrelated tasks. A 7B model fine-tuned on SQL generation may lose general reasoning. LoRA mitigates this but doesn’t eliminate it. You need held-out eval sets that cover both the target task and regression benchmarks.

Data quality dominates fine-tuning outcomes. 1,000 clean, diverse examples beat 10,000 noisy ones. Few-shot is more forgiving of label noise because the model sees the pattern at inference time.

Which to choose: verdict by use case

Start with few-shot when:

• Exploring a new task, unknown requirements
• Low volume (< 100K requests/month)
• Task changes weekly — classification schemas, extraction fields, tone guidelines
• You need zero infrastructure investment
• Prototyping for stakeholder demos

Switch to fine-tuning when:

• Volume exceeds ~500K requests/month and latency matters
• Few-shot accuracy plateaus below requirements despite prompt iteration
• You need consistent structured output (JSON schema adherence, function calling)
• Domain terminology or reasoning patterns are too complex for context window
• You’re deploying to edge or constrained environments (smaller model, no examples)
• Regulatory requirements demand reproducible, auditable model behavior

Hybrid approach (often optimal): Fine-tune a small model (7B–8B) for the core task. Use few-shot on top for dynamic instructions, user preferences, or context that changes per request. The fine-tuned model handles the invariant pattern; the prompt handles the variable part.

# hybrid: fine-tuned base + dynamic few-shot
messages = [
    {"role": "system", "content": "You are a support classifier (fine-tuned)."},
    {"role": "user", "content": "Current policy: refunds within 30 days only."},
    {"role": "user", "content": f"Ticket: \"{user_input}\"\nLabel:"},
]

Decision checklist:

1. Can you articulate the task as a clear input-output mapping with < 50 diverse examples? → few-shot
2. Do you need < 100ms p99 latency at > 100 RPS? → fine-tune
3. Is the task schema stable for 3+ months? → fine-tune
4. Do you have eval infrastructure or budget to build it? → fine-tune
5. Is the team comfortable owning model serving? → fine-tune

If you answered “no” to any of 2–5, stay on few-shot until the pain justifies the investment. Most teams overestimate how soon they need fine-tuning. The few-shot ceiling is higher than you think — especially with 128K+ context windows and prompt caching.