Prompt engineering basics: a beginner's guide

Prompt engineering basics start with understanding that LLMs are completion engines, not reasoning engines. Every token you send shapes the probability distribution of what comes next. The difference between a fragile prompt and a production-ready one isn’t clever phrasing — it’s explicit structure, constrained output formats, and systematic evaluation. This guide walks through the techniques that actually move the needle.

Start with the system prompt

The system prompt sets the behavioral contract. It runs before any user input and persists across the conversation. Treat it like infrastructure code: version it, test it, and keep it minimal.

SYSTEM_PROMPT = """You are a senior backend engineer. 
Write production-ready Python 3.11+ code. 
Prefer standard library over dependencies. 
Include type hints and docstrings. 
Return only the code block — no explanations unless asked."""

Pitfall: overloading the system prompt with task-specific instructions. Those belong in the user message or few-shot examples. The system prompt should define persona and constraints, not task logic.

Use structured output formats

Unstructured text is hard to parse reliably. Force the model into a machine-readable format — JSON, YAML, or a strict markdown schema — and validate it.

import json
from pydantic import BaseModel, Field, ValidationError
from typing import Literal

class CodeReview(BaseModel):
    severity: Literal["blocker", "major", "minor", "nit"]
    file: str
    line: int
    message: str
    suggestion: str | None = None

OUTPUT_SCHEMA = CodeReview.model_json_schema()

USER_PROMPT = f"""Review the following diff. 
Return a JSON array matching this schema exactly:
{json.dumps(OUTPUT_SCHEMA, indent=2)}

Diff:
```diff
- def process(items):
+ def process(items: list[str]) -> list[str]:
     return [x.strip() for x in items if x]
```"""

Tradeoff: structured output costs tokens and latency. For high-volume paths, consider a lighter delimiter-based format (e.g., ### SEVERITY: blocker\n### FILE: ...) and parse with regex. Validate either way.

Few-shot prompting with deliberate examples

Zero-shot works for simple classification. For anything with nuance — code style, tone, multi-step reasoning — provide 3–5 diverse examples. The examples teach the pattern better than instructions.

FEW_SHOT_EXAMPLES = [
    {
        "input": "def add(a, b): return a + b",
        "output": {"severity": "nit", "message": "Missing type hints", "suggestion": "def add(a: int, b: int) -> int:"}
    },
    {
        "input": "password = 'secret123'",
        "output": {"severity": "blocker", "message": "Hardcoded credential", "suggestion": "Use environment variable"}
    },
    {
        "input": "for i in range(len(items)): print(items[i])",
        "output": {"severity": "minor", "message": "Non-idiomatic iteration", "suggestion": "for item in items: print(item)"}
    },
]

Pitfall: examples that are too similar. The model learns the common structure, not the decision boundary. Include edge cases, false positives you want to avoid, and examples where the correct answer is “no issue.”

Chain-of-thought for multi-step tasks

When the task requires reasoning — debugging, refactoring, architecture decisions — force the model to show work. The reasoning tokens become part of the context for the final answer.

COT_PROMPT = """Analyze this performance issue step by step.
Think through: 
1. What the code does
2. Where the bottleneck likely is
3. Alternative approaches
4. Tradeoffs of each

Then provide the optimized version.

Code:
```python
def find_duplicates(items: list[int]) -> list[int]:
    seen = set()
    dupes = []
    for item in items:
        if item in seen:
            dupes.append(item)
        else:
            seen.add(item)
    return dupes
```"""

Tradeoff: chain-of-thought increases latency and token usage 3–10x. Use it for planning and complex generation; skip it for classification, extraction, and simple transforms. You can also request reasoning internally and emit only the final answer:

HIDDEN_COT_PROMPT = """Reason through this internally, then output ONLY the fixed code."""

Constrained decoding via logit bias and stop sequences

For classification and enum-like outputs, constrain the vocabulary. This is more reliable than prompting “only output A, B, or C.”

# OpenAI-compatible parameters
params = {
    "logit_bias": {
        "1234": 100,  # token ID for "blocker"
        "5678": 100,  # token ID for "major"
        "9012": 100,  # token ID for "minor"
        "3456": 100,  # token ID for "nit"
    },
    "stop": ["\n\n", "###"],
    "max_tokens": 10,
}

You need the tokenizer to map strings to token IDs. For portable prompts, prefer stop sequences and post-hoc validation over logit bias.

Retrieval-augmented context for domain knowledge

Prompt engineering basics break down when the model lacks domain context. Inject relevant documentation, schema definitions, or prior art at query time rather than stuffing everything into the system prompt.

def build_context(query: str, vector_store, k: int = 5) -> str:
    results = vector_store.similarity_search(query, k=k)
    chunks = []
    for i, doc in enumerate(results):
        chunks.append(f"### Source {i+1} ({doc.metadata['source']})\n{doc.page_content}")
    return "\n\n".join(chunks)

RAG_PROMPT = """Answer using only the provided context. 
If the answer isn't in the context, say "I don't know."

Context:
{context}

Question: {question}"""

Pitfall: context window overflow. Summarize or truncate retrieved chunks to fit your budget. A 128k context window doesn’t mean you should fill it — attention dilutes, and latency scales superlinearly.

Evaluation: the missing loop

You cannot engineer prompts without evaluation. Build a test set of 50–200 representative inputs with expected outputs. Run it on every prompt change.

from dataclasses import dataclass
from typing import Callable

@dataclass
class TestCase:
    input: str
    expected: dict
    tags: list[str]

def evaluate(prompt_fn: Callable[[str], dict], test_cases: list[TestCase]) -> dict:
    results = {"pass": 0, "fail": 0, "errors": []}
    for tc in test_cases:
        try:
            output = prompt_fn(tc.input)
            if matches_expected(output, tc.expected):
                results["pass"] += 1
            else:
                results["fail"] += 1
                results["errors"].append({"input": tc.input, "expected": tc.expected, "got": output})
        except Exception as e:
            results["fail"] += 1
            results["errors"].append({"input": tc.input, "error": str(e)})
    return results

def matches_expected(output: dict, expected: dict) -> bool:
    # Custom logic: exact match, semantic equivalence, subset, etc.
    return output.get("severity") == expected.get("severity")

Tag test cases by category (security, style, performance, false-positive-prone). Regression in one tag tells you exactly what broke.

Iterative refinement with critique prompts

Use a second prompt to critique the first model’s output. This catches hallucinations, style violations, and missed requirements without human review.

CRITIQUE_PROMPT = """You are a strict code reviewer. 
Check the proposed fix against the original issue.
Return JSON: {{"pass": bool, "issues": string[]}}

Original issue: {issue}
Proposed fix: {fix}"""

Run this in the same request chain (cheaper) or as a separate validation step (more reliable). For production pipelines, the critique model can be smaller and faster than the generator.

Version control your prompts

Prompts are code. Store them in version control, not in environment variables or database rows.

prompts/
├── code_review/
│   ├── v1_system.md
│   ├── v1_user_template.md
│   ├── v2_system.md          # added security rules
│   ├── v2_user_template.md
│   └── test_cases.jsonl
├── sql_generation/
│   └── ...

Tag prompt versions with the model they were tuned for. A prompt optimized for GPT-4o often degrades on Claude 3.5 Sonnet or Llama 3.1 405B. When you switch models, re-run evaluation.

Common pitfalls summary

Putting it together: a production prompt template

from string import Template
from pydantic import BaseModel
import json

class PromptTemplate(BaseModel):
    system: str
    user_template: Template
    output_schema: dict
    stop_sequences: list[str] = []
    max_tokens: int = 2048
    temperature: float = 0.1

CODE_REVIEW_V2 = PromptTemplate(
    system="""You are a senior backend engineer. 
Write production-ready Python 3.11+ code. 
Prefer standard library. Include type hints.
Return ONLY valid JSON matching the schema.""",
    user_template=Template("""Review this diff for security, performance, and style issues.
Return a JSON array of findings. Each finding must match the schema.

Schema:
$schema

Diff:
```diff
$diff
```"""),
    output_schema=CodeReview.model_json_schema(),
    stop_sequences=["```"],
    max_tokens=1024,
    temperature=0.0,
)

def render_prompt(template: PromptTemplate, **kwargs) -> dict:
    return {
        "messages": [
            {"role": "system", "content": template.system},
            {"role": "user", "content": template.user_template.substitute(**kwargs)},
        ],
        "response_format": {"type": "json_schema", "json_schema": template.output_schema},
        "stop": template.stop_sequences,
        "max_tokens": template.max_tokens,
        "temperature": template.temperature,
    }

This structure — template + schema + parameters + eval set — is the unit of deployment. When you route requests through a gateway that supports per-model parameter overrides and automatic fallback, you can swap models without rewriting prompt logic. The prompt engineering basics stay the same; only the tuning changes.