Estimating LLM API costs before you make the call

You can estimate llm api cost before request by combining exact token counts with live provider pricing. Building this check into your service avoids surprise bills and lets you reject oversized prompts at the edge. The pattern takes six steps: match tokenizer, count tokens, load prices, compute, gate, and reconcile.

Step 1: Identify the target model and its tokenizer

Tokenizers are not interchangeable. Counting with cl100k_base when the backend runs Llama 3 will undershoot real usage by 10–30% on code and non-English text. A mismatch means your cost guard fires too late. Pull the encoder that ships with the model.

For OpenAI-compatible models, tiktoken is the reference implementation:

import tiktoken

def get_encoder(model: str):
    try:
        return tiktoken.encoding_for_model(model)
    except KeyError:
        # fallback only for internal aliases; prefer explicit mapping
        return tiktoken.get_encoding("cl100k_base")

For open-weight models, load the exact tokenizer from Hugging Face. Guessing the family is not enough; Llama 3 and Mixtral use different vocab sizes and merge rules.

from tokenizers import Tokenizer

tok = Tokenizer.from_pretrained("meta-llama/Llama-3-70b-instruct")
# len(tok.encode("...").ids) is your count

Maintain a static map from model id to tokenizer loader so you never guess at runtime. If you proxy through a gateway that addresses 240+ models behind one endpoint, you still need the per-model tokenizer locally; the gateway does not count tokens for you.

Step 2: Count prompt and maximum completion tokens

Count the system, user, and assistant turns as a single concatenated string exactly as you will send it, including whitespace, JSON formatting, and tool schemas. Then read max_tokens from your request config as the upper bound for output.

def count_tokens(text: str, encoder) -> int:
    if hasattr(encoder, "encode"):
        return len(encoder.encode(text))
    return len(encoder.encode(text).ids)  # HF Tokenizer

prompt_text = build_prompt(messages)
prompt_tokens = count_tokens(prompt_text, enc)
max_completion = req.get("max_tokens", 1024)

Never treat max_tokens as the expected cost. It is the ceiling. If your traffic has a typical completion length, track a rolling average and estimate with that instead of the ceiling to get a realistic number for budgeting, but keep the ceiling for hard limits.

Vision and audio inputs break the text tokenizer. A 1024x1024 image on GPT-4o costs a fixed 765 tokens plus a per-tile cost; count those separately or your estimate llm api cost before request will be wrong by orders of magnitude.

Step 3: Retrieve current per-token pricing

Provider rates change. Hard-coding prices in source is a liability. Store them in a versioned JSON file or fetch from a pricing API at deploy time.

{
  "gpt-4o": {"input": 5.0, "output": 15.0, "per": 1000000},
  "gpt-4o-mini": {"input": 0.15, "output": 0.6, "per": 1000000},
  "llama-3-70b-instruct": {"input": 0.5, "output": 0.8, "per": 1000000}
}

Rates are per million tokens for most vendors; confirm the denominator. Batch APIs often cut rates 50% but add latency. Prompt caching introduces a cached_input rate at a discount (often 10–50% of input). If you use cache-control hints, record that discount in the pricing file.

Step 4: Compute the pre-request cost estimate

Multiply token counts by rates and sum. Use the ceiling for completion to fail safe.

def estimate_cost(model: str, prompt_tok: int, completion_tok: int, pricing: dict) -> float:
    rate = pricing[model]
    in_cost = prompt_tok * rate["input"] / rate["per"]
    out_cost = completion_tok * rate["output"] / rate["per"]
    return in_cost + out_cost

est = estimate_cost("gpt-4o", prompt_tokens, max_completion, pricing)

If you forward cache-control hints, subtract the cached portion. This is how you estimate llm api cost before request with cache awareness:

cached_tok = cache_policy.tokens_for(model, prompt_text)
uncached_tok = max(0, prompt_tokens - cached_tok)
in_cost = (uncached_tok * rate["input"] + cached_tok * rate.get("cached_input", rate["input"])) / rate["per"]

Add image or audio token costs the same way. The goal is a single number you can compare to a threshold in microseconds.

Step 5: Gate the request with a budget check

Put the estimate behind your API boundary. Reject before calling the model.

MAX_PER_CALL = 0.02  # $0.02

if est > MAX_PER_CALL:
    raise BudgetExceeded(f"estimate ${est:.4f} > limit ${MAX_PER_CALL:.4f}")

resp = client.chat.completions.create(model=model, messages=messages, max_tokens=max_completion)

For batch jobs, sum estimates across the queue and block the batch if the projected spend exceeds the daily cap. This converts an open-ended loop into a bounded one. Wrap the check in a decorator so every entry point gets it:

def budget_guard(limit):
    def wrap(fn):
        def inner(model, messages, **kw):
            est = estimate_cost(model, count_prompt(messages), kw.get("max_tokens", 1024), pricing)
            if est > limit:
                raise BudgetExceeded(est)
            return fn(model, messages, **kw)
        return inner
    return wrap

Step 6: Reconcile estimate with actual usage

After the call, the provider returns exact token counts. Compare them to your prediction and log the delta.

usage = resp.usage
actual = (usage.prompt_tokens * rate["input"] + usage.completion_tokens * rate["output"]) / rate["per"]
log.info("cost_estimate=%f actual=%f delta=%f", est, actual, actual - est)

If you route through n4n.ai, its per-token usage metering reports exact counts and respects any cache-control hints you forwarded, so your cached-input discount can be validated against the bill.

Wire the delta into an alert: if your estimator is consistently low by more than 5%, your tokenizer mapping or image token math is wrong.

Common pitfalls when you estimate llm api cost before request

Tool calls inflate prompt size silently. A function schema serialized into the request can add hundreds of tokens; count it. Streaming does not change token counts but makes it tempting to skip the ceiling—don’t. Multi-region pricing means the same model id can cost differently depending on the datacenter; key your pricing file by region if you route globally. Finally, provider rate changes without notice; a stale price file is the most common cause of blown budgets.

Verify your pipeline

Success means three things. First, a unit test with known text returns the same token count as the provider’s tokenizer playground. Second, a request that would cost more than your limit raises BudgetExceeded and never hits the network—verify with a mock client. Third, the post-call log shows actual cost within 5% of estimate for both cached and uncached paths over a sample of real traffic. Run these checks in CI with a frozen pricing file, and refresh the file on a schedule.

That loop—estimate, gate, reconcile—turns LLM spend from a mystery into a configured constant.