10 prompt engineering techniques that actually work

Most prompt engineering advice is either too vague to implement or too academic to ship. The techniques below are the ones that survive contact with production traffic — they reduce hallucination, improve structure, and make model behavior predictable enough to build on. If you’re evaluating prompt engineering techniques for a real system, start here.

1. Chain-of-thought prompting

Force the model to show its work before giving the final answer. This single change improves accuracy on multi-step reasoning tasks by 15-30% across most benchmarks. The key is making the reasoning explicit in the output, not just hoping the model “thinks harder.”

SYSTEM = """You are a careful reasoning engine. For each question:
1. Break the problem into sub-problems
2. Solve each sub-problem step by step
3. State your final answer clearly"""

USER = """If a train leaves Chicago at 60mph and another leaves 
St. Louis at 70mph toward each other, and the cities are 300 miles 
apart, when do they meet? Show your reasoning."""

The model emits intermediate steps you can log, audit, or feed into downstream validation. For production, wrap the reasoning in a structured block (XML tags work well) so you can parse it programmatically and discard or display it separately from the final answer.

2. Few-shot prompting with diverse examples

Zero-shot fails on nuanced formatting or domain-specific logic. Few-shot works — but only if your examples cover the actual distribution of edge cases, not just the happy path. Include: a standard case, a tricky formatting case, an adversarial/ambiguous case, and a refusal case.

{
  "examples": [
    {"input": "Extract entities: Apple released iPhone 15", "output": '{"org": "Apple", "product": "iPhone 15"}'},
    {"input": "Extract entities: The bank approved the loan", "output": '{"org": null, "product": null}'},
    {"input": "Extract entities: Microsoft's Azure revenue grew 20%", "output": '{"org": "Microsoft", "product": "Azure"}'},
    {"input": "Extract entities: I like pizza", "output": '{"org": null, "product": null}'}
  ]
}

Curate 8-12 examples. More than that yields diminishing returns and burns context window. Rotate examples per request if you have a larger pool — this prevents the model from overfitting to a fixed demonstration set.

3. Structured output formatting with schemas

Don’t ask for JSON and hope. Define a schema, include it in the prompt, and validate the response. This eliminates the “almost valid JSON” failure mode that breaks downstream parsers.

import json
from pydantic import BaseModel, Field

class Extraction(BaseModel):
    organization: str | None = Field(description="Company or org name")
    product: str | None = Field(description="Product or service name")
    confidence: float = Field(ge=0, le=1)

SCHEMA = Extraction.model_json_schema()

PROMPT = f"""Extract entities. Respond ONLY with valid JSON matching this schema:
{json.dumps(SCHEMA, indent=2)}

Text: {{text}}"""

On validation failure, retry once with an error message appended: “Your previous response failed validation: {error}. Output only corrected JSON.” This two-pass approach catches 95%+ of formatting errors without human intervention.

4. System prompt separation

Mixing instructions into the user message creates prompt injection surface area and makes behavior inconsistent across turns. Keep system-level instructions (role, tone, constraints, output format) in a dedicated system message. Put only the variable task input in the user message.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": user_input}
]

This also lets you version and A/B test system prompts independently of user-facing copy. When you need to inject dynamic context (retrieved docs, user profile), add it as a separate “context” message with a clear delimiter, not by concatenating into the system prompt.

5. Temperature and sampling control

Temperature is not a creativity knob — it’s a determinism knob. For extraction, classification, and coding tasks, use temperature=0 (or top_p=0.1). For brainstorming or open-ended generation, 0.7-0.9. Never leave it at the default.

COMPLETION_PARAMS = {
    "extraction": {"temperature": 0, "top_p": 0.1, "max_tokens": 500},
    "summarization": {"temperature": 0.3, "top_p": 0.9, "max_tokens": 1000},
    "creative": {"temperature": 0.8, "top_p": 0.95, "max_tokens": 2000},
}

Set seed for reproducibility when debugging. Some providers (OpenAI, Anthropic) honor it; others ignore it. Log the actual parameters used per request — you’ll need this when a production issue traces back to a sampling change.

6. Prompt chaining and task decomposition

Complex tasks fail when crammed into one prompt. Decompose into a pipeline: each step has a single responsibility, validated output, and a clear contract. This also lets you route steps to different models (cheap for classification, expensive for synthesis).

async def analyze_contract(text: str) -> Analysis:
    # Step 1: Classify document type (cheap model)
    doc_type = await classify(text, model="haiku")
    
    # Step 2: Extract clauses (specialized prompt)
    clauses = await extract_clauses(text, doc_type, model="sonnet")
    
    # Step 3: Flag risks (reasoning model)
    risks = await flag_risks(clauses, model="opus")
    
    # Step 4: Synthesize report
    return await synthesize_report(doc_type, clauses, risks, model="sonnet")

Each step can be retried, cached, or swapped independently. The intermediate outputs are also valuable for observability — you know exactly where the pipeline broke.

7. Retrieval-augmented prompting

When the task requires knowledge beyond the model’s training cutoff or your proprietary data, retrieve first, then prompt. The prompt template matters: cite sources inline, handle “no relevant docs” gracefully, and bound the context window.

RAG_PROMPT = """Answer the question using ONLY the provided context.
If the context doesn't contain the answer, say "I don't know."
Cite sources like [doc_1], [doc_2] inline.

Context:
{context}

Question: {question}

Answer:"""

Chunk size: 512-1024 tokens with 10-20% overlap. Retrieve top-k=5-8, then rerank with a cross-encoder if latency budget allows. For n4n.ai users, the gateway’s automatic fallback across 240+ models means your RAG pipeline stays up even when your primary provider degrades — just ensure your prompt template is provider-agnostic.

8. Self-consistency and majority voting

For tasks with a verifiable correct answer (math, multiple choice, code correctness), run the same prompt N times at temperature > 0 and take the majority vote. This trades latency for accuracy.

async def self_consistency(prompt: str, n: int = 5) -> str:
    responses = await asyncio.gather(*[
        complete(prompt, temperature=0.7) for _ in range(n)
    ])
    # Parse answers, count frequencies
    answers = [extract_answer(r) for r in responses]
    return Counter(answers).most_common(1)[0][0]

N=5 is the sweet spot for most tasks. For code generation, execute each candidate and vote on passing tests rather than text similarity — this catches semantically equivalent but syntactically different solutions.

9. Negative constraints and guardrails

Models follow positive instructions better than negative ones, but explicit prohibitions prevent specific failure modes. List what the output must NOT contain: no apologies, no markdown unless requested, no assumptions beyond context, no PII.

GUARDRAILS = """Constraints:
- Do not include any explanatory text outside the JSON object
- Do not infer information not present in the input
- Do not use markdown formatting
- If uncertain, set confidence < 0.5 and explain in reasoning field
- Never output PII (emails, phones, SSNs) — redact as [REDACTED]"""

Pair with a post-generation validator that checks for forbidden patterns (regex for PII, JSON schema for structure, keyword blocklist for tone). Reject and retry on violation — this is cheaper than cleaning up bad data downstream.

10. Iterative refinement with feedback loops

The first prompt is a draft. Build a feedback loop: log inputs, outputs, and human ratings (or automated evals). Use the failures to write better few-shot examples, tighten constraints, or adjust the system prompt. Treat prompts as code — version control them, review changes, run regression tests.

# eval_harness.py
TEST_CASES = [
    {"input": "...", "expected": {...}, "min_score": 0.9},
    # ...
]

async def evaluate(prompt_version: str) -> dict:
    results = []
    for tc in TEST_CASES:
        output = await run_prompt(prompt_version, tc["input"])
        score = score_output(output, tc["expected"])
        results.append({"case": tc, "score": score, "output": output})
    return {"pass_rate": mean(r["score"] >= tc["min_score"] for r, tc in zip(results, TEST_CASES))}

Run this on every prompt change. A 2% regression on a core task catches problems before they hit users. The teams shipping reliable LLM features aren’t the ones with the cleverest prompts — they’re the ones with the tightest eval loops.

Summary table

Pick three to start: structured output, system separation, and an eval harness. The rest compound from there.