How much text fits in a 128k token context window?

If you’re building on LLMs, you’ve asked yourself how much text is 128k tokens at least once. The answer depends on the tokenizer, the language, and the content itself — but you can measure it precisely for your use case. This guide walks through the steps to estimate, calculate, and verify context capacity for any model exposing a 128k window.

Step 1: Understand what a token actually represents

Tokens are not words. They’re subword units produced by a tokenizer (BPE, WordPiece, Unigram). English averages ~1.3 tokens per word. Code averages ~1.5–2 tokens per word due to punctuation and identifiers. Non-Latin scripts often run 2–4 tokens per character.

A 128k context window means the model can attend to 131,072 tokens total — prompt plus completion. If your prompt consumes 100k tokens, you have ~31k left for the response.

Quick reference for English prose:

These are rules of thumb. The only reliable number comes from running your actual text through the model’s tokenizer.

Step 2: Pick the right tokenizer for your model

Each model family uses a different tokenizer. Using the wrong one gives you systematically wrong counts.

Install the relevant library:

# OpenAI models
pip install tiktoken

# Hugging Face models (Llama, Gemma, etc.)
pip install transformers tokenizers

# Mistral
pip install mistral-common

# Cohere
pip install cohere

Step 3: Count tokens programmatically

Here are minimal, runnable snippets for the most common model families.

OpenAI models (GPT-4o, GPT-4, GPT-3.5)

import tiktoken

def count_tokens_openai(text: str, model: str = "gpt-4o") -> int:
    """Return token count for an OpenAI model."""
    try:
        encoding = tiktoken.encoding_for_model(model)
    except KeyError:
        # Fallback for newer models not yet in tiktoken
        encoding = tiktoken.get_encoding("o200k_base")
    return len(encoding.encode(text))

# Example
text = "Your 128k context window holds approximately 98,000 English words."
print(f"Tokens: {count_tokens_openai(text, 'gpt-4o')}")

Llama 3 / 3.1 (and other Hugging Face models)

from transformers import AutoTokenizer

def count_tokens_hf(text: str, model_id: str = "meta-llama/Meta-Llama-3-8B") -> int:
    """Return token count for a Hugging Face model."""
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    return len(tokenizer.encode(text))

# Example
text = "Your 128k context window holds approximately 98,000 English words."
print(f"Tokens: {count_tokens_hf(text, 'meta-llama/Meta-Llama-3-8B')}")

Mistral / Mixtral

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer

def count_tokens_mistral(text: str, version: str = "v3") -> int:
    """Return token count for Mistral models."""
    tokenizer = MistralTokenizer.from_version(version)
    return len(tokenizer.encode(text).tokens)

# Example
text = "Your 128k context window holds approximately 98,000 English words."
print(f"Tokens: {count_tokens_mistral(text, 'v3')}")

Cohere Command R / R+

import cohere

def count_tokens_cohere(text: str, model: str = "command-r-plus") -> int:
    """Return token count for Cohere models."""
    client = cohere.Client()  # Requires COHERE_API_KEY
    response = client.tokenize(text=text, model=model)
    return len(response.tokens)

# Example
text = "Your 128k context window holds approximately 98,000 English words."
print(f"Tokens: {count_tokens_cohere(text, 'command-r-plus')}")

Step 4: Account for chat formatting overhead

Raw text counts aren’t enough. Chat templates add special tokens for roles, message boundaries, and system prompts. These consume budget.

OpenAI chat format overhead

Each message adds ~4 tokens (role + formatting). The final assistant priming adds ~3 tokens.

import tiktoken

def count_chat_tokens_openai(messages: list[dict], model: str = "gpt-4o") -> int:
    """Count tokens for a full chat conversation including formatting."""
    encoding = tiktoken.encoding_for_model(model)
    tokens = 0
    for msg in messages:
        # Each message: <|im_start|>{role}\n{content}<|im_end|>\n
        tokens += 4  # overhead per message
        for key, value in msg.items():
            tokens += len(encoding.encode(str(value)))
    tokens += 3  # assistant priming
    return tokens

# Example
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Summarize this document: " + "x" * 50000},
]
print(f"Chat tokens: {count_chat_tokens_openai(messages, 'gpt-4o')}")

Llama 3 chat template

from transformers import AutoTokenizer

def count_chat_tokens_llama3(messages: list[dict], model_id: str = "meta-llama/Meta-Llama-3-8B-Instruct") -> int:
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    # apply_chat_template adds special tokens; tokenize=False returns string
    rendered = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    return len(tokenizer.encode(rendered))

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Summarize this document: " + "x" * 50000},
]
print(f"Chat tokens: {count_chat_tokens_llama3(messages)}")

Step 5: Measure real-world content types

Token density varies wildly by content type. Run your actual data through the tokenizer.

import tiktoken

encoding = tiktoken.encoding_for_model("gpt-4o")

samples = {
    "english_prose": " ".join(["The quick brown fox jumps over the lazy dog."] * 1000),
    "python_code": "def fibonacci(n):\n    a, b = 0, 1\n    for _ in range(n):\n        yield a\n        a, b = b, a + b\n" * 500,
    "json_data": '{"users": [' + ','.join([f'{{"id":{i},"name":"User {i}","email":"user{i}@example.com"}}' for i in range(1000)]) + ']}',
    "chinese_text": "这是一个测试句子。" * 2000,
    "mixed_code_comments": "// Process user data\nfunction processUser(user) {\n  // Validate input\n  if (!user || !user.id) return null;\n  // Transform\n  return {\n    id: user.id,\n    name: user.name.toUpperCase(),\n    active: true\n  };\n}\n" * 300,
}

for name, text in samples.items():
    tokens = len(encoding.encode(text))
    words = len(text.split())
    ratio = tokens / words if words else 0
    print(f"{name:20s} | {tokens:>7,} tokens | {words:>7,} words | {ratio:.2f} tokens/word")

Typical output on GPT-4o tokenizer:

english_prose        |   16,243 tokens |   11,000 words | 1.48 tokens/word
python_code          |   28,912 tokens |   10,500 words | 2.75 tokens/word
json_data            |   42,105 tokens |    8,000 words | 5.26 tokens/word
chinese_text         |   24,000 tokens |    8,000 chars | 3.00 tokens/char
mixed_code_comments  |   18,450 tokens |    7,200 words | 2.56 tokens/word

JSON and code are token-expensive. Chinese characters average ~2–3 tokens each. Plan accordingly.

Step 6: Calculate your effective context budget

Subtract all fixed overhead from 128k to get your usable budget.

def calculate_budget(
    model_context: int = 131072,
    system_prompt_tokens: int = 0,
    few_shot_examples_tokens: int = 0,
    chat_overhead_per_message: int = 4,
    num_messages: int = 0,
    reserved_completion_tokens: int = 4096,
) -> int:
    """Return usable input tokens after overhead."""
    overhead = (
        system_prompt_tokens
        + few_shot_examples_tokens
        + chat_overhead_per_message * num_messages
        + reserved_completion_tokens
    )
    return max(0, model_context - overhead)

# Example: GPT-4o with system prompt, 3-shot, 5-message conversation, 4k reserved for output
budget = calculate_budget(
    system_prompt_tokens=500,
    few_shot_examples_tokens=2000,
    num_messages=5,
    reserved_completion_tokens=4096,
)
print(f"Usable input tokens: {budget:,}")
# Output: Usable input tokens: 124,472

Reserve completion tokens explicitly. If you need 8k output, reserve 8k. The model will stop at the context limit mid-sentence otherwise.

Step 7: Implement runtime guardrails

Count tokens before sending requests. Fail fast rather than hitting a 400 error.

import tiktoken
from dataclasses import dataclass
from typing import Optional

@dataclass
class ContextBudget:
    model_context: int
    reserved_completion: int
    system_tokens: int = 0
    few_shot_tokens: int = 0
    message_overhead: int = 4

    @property
    def max_input_tokens(self) -> int:
        return self.model_context - self.reserved_completion - self.system_tokens - self.few_shot_tokens

    def check_fit(self, messages: list[dict], model: str = "gpt-4o") -> tuple[bool, int, int]:
        """Return (fits, used_tokens, remaining_tokens)."""
        encoding = tiktoken.encoding_for_model(model)
        used = 0
        for msg in messages:
            used += self.message_overhead
            for v in msg.values():
                used += len(encoding.encode(str(v)))
        used += 3  # assistant priming
        remaining = self.max_input_tokens - used
        return remaining >= 0, used, remaining

# Usage
budget = ContextBudget(
    model_context=131072,
    reserved_completion=4096,
    system_tokens=500,
    few_shot_tokens=2000,
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "x" * 100000},
]

fits, used, remaining = budget.check_fit(messages, "gpt-4o")
if not fits:
    raise ValueError(f"Prompt exceeds budget by {abs(remaining):,} tokens. Used: {used:,}, Limit: {budget.max_input_tokens:,}")
print(f"OK: {used:,} tokens used, {remaining:,} remaining")

Step 8: Handle streaming and dynamic contexts

In multi-turn conversations, you must truncate history to stay within budget. Implement a sliding window or summarization strategy.

def truncate_history(
    messages: list[dict],
    budget: ContextBudget,
    model: str = "gpt-4o",
    keep_system: bool = True,
    keep_last_n: int = 2,
) -> list[dict]:
    """Remove oldest messages until conversation fits."""
    encoding = tiktoken.encoding_for_model(model)
    
    def count_msgs(msgs: list[dict]) -> int:
        total = 0
        for m in msgs:
            total += budget.message_overhead
            for v in m.values():
                total += len(encoding.encode(str(v)))
        total += 3
        return total
    
    # Always keep system message if present
    system_msg = messages[0] if keep_system and messages and messages[0]["role"] == "system" else None
    conversation = messages[1:] if system_msg else messages[:]
    
    while conversation and count_msgs([system_msg] + conversation) > budget.max_input_tokens:
        # Remove oldest non-kept message
        if len(conversation) <= keep_last_n:
            break  # Can't truncate further without losing required messages
        conversation.pop(0)
    
    result = []
    if system_msg:
        result.append(system_msg)
    result.extend(conversation)
    return result

# Test
long_conversation = [
    {"role": "system", "content": "System prompt."},
] + [
    {"role": "user" if i % 2 == 0 else "assistant", "content": f"Message {i} " + "x" * 5000}
    for i in range(20)
]

truncated = truncate_history(long_conversation, budget, "gpt-4o")
fits, used, remaining = budget.check_fit(truncated, "gpt-4o")
print(f"Kept {len(truncated)} messages, {used:,} tokens, {remaining:,} remaining")

Step 9: Verify with actual API calls

Token counting is deterministic, but provider implementations can differ. Verify against the real API.

import os
from openai import OpenAI

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def verify_token_count(model: str, messages: list[dict]) -> dict:
    """Send a minimal completion to get actual usage from the API."""
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        max_tokens=1,  # Minimal completion
        temperature=0,
    )
    return {
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
        "total_tokens": response.usage.total_tokens,
    }

# Compare local count vs API count
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Count the tokens in this message."},
]

local_count = count_chat_tokens_openai(messages, "gpt-4o")
api_usage = verify_token_count("gpt-4o", messages)

print(f"Local count:  {local_count}")
print(f"API count:    {api_usage['prompt_tokens']}")
print(f"Difference:   {abs(local_count - api_usage['prompt_tokens'])}")

Expected difference: 0–10 tokens (usually 0). If you see systematic offsets, your chat template implementation differs from the provider’s.

Step 10: Monitor production token usage

Log token consumption per request. Alert on trends approaching limits.

import time
import logging
from dataclasses import dataclass, asdict
from typing import Optional

@dataclass
class TokenMetrics:
    request_id: str
    model: str
    prompt_tokens: int
    completion_tokens: int
    total_tokens: int
    context_limit: int
    utilization_pct: float
    latency_ms: int
    timestamp: float

def log_token_metrics(metrics: TokenMetrics) -> None:
    """Log structured metrics for observability."""
    logging.info("token_usage", extra=asdict(metrics))
    
    # Alert if utilization > 90%
    if metrics.utilization_pct > 90:
        logging.warning(
            "high_context_utilization",
            extra={
                "request_id": metrics.request_id,
                "utilization_pct": metrics.utilization_pct,
                "remaining_tokens": metrics.context_limit - metrics.total_tokens,
            }
        )

# Wrapper for instrumented calls
def tracked_completion(client: OpenAI, model: str, messages: list[dict], **kwargs) -> tuple:
    request_id = f"req_{int(time.time() * 1000)}"
    start = time.perf_counter()
    
    response = client.chat.completions.create(model=model, messages=messages, **kwargs)
    
    latency_ms = int((time.perf_counter() - start) * 1000)
    usage = response.usage
    context_limit = 131072 if "128k" in model or model in ("gpt-4o", "gpt-4-turbo") else 8192
    
    metrics = TokenMetrics(
        request_id=request_id,
        model=model,
        prompt_tokens=usage.prompt_tokens,
        completion_tokens=usage.completion_tokens,
        total_tokens=usage.total_tokens,
        context_limit=context_limit,
        utilization_pct=round(usage.total_tokens / context_limit * 100, 2),
        latency_ms=latency_ms,
        timestamp=time.time(),
    )
    log_token_metrics(metrics)
    return response, metrics

Verification checklist

Before deploying a 128k-context workflow, confirm:

• Token counts match API-reported usage within ±10 tokens for representative samples
• Chat template overhead is accounted for (system prompt, few-shot, message formatting)
• Completion budget is reserved and enforced client-side
• Truncation strategy preserves required context (system prompt, recent turns)
• Non-English content is measured with actual tokenizer, not estimated
• Structured logging captures utilization per request
• Alerts fire at 90% context utilization

How much text is 128k tokens for your workload?

The only honest answer: run your data through the tokenizer. The code above gives you everything needed to measure, budget, and guardrail a 128k context window in production.