Switching from Fireworks AI to n4n: what changes

Switching from Fireworks AI to n4n changes more than your HTTP base URL. The two systems expose similar OpenAI-compatible chat endpoints, but they differ in model routing, billing granularity, and how they behave when an upstream provider degrades. If you are migrating a production workload, treat this as a re-platforming of your inference layer, not a drop-in swap.

API surface and compatibility

Fireworks AI ships an OpenAI-compatible REST API. You POST to https://api.fireworks.ai/inference/v1/chat/completions with a model string like accounts/fireworks/models/llama-v3p1-8b-instruct. n4n mirrors the same shape at its gateway endpoint, accepting the standard messages, temperature, tools, and stream fields. The first concrete change is the model identifier namespace.

# Fireworks
from openai import OpenAI
import os

fw = OpenAI(
    base_url="https://api.fireworks.ai/inference/v1",
    api_key=os.environ["FW_KEY"],
)
resp = fw.chat.completions.create(
    model="accounts/fireworks/models/llama-v3p1-8b-instruct",
    messages=[{"role": "user", "content": "ping"}],
    stream=False,
)

# n4n
n4n = OpenAI(
    base_url="https://api.n4n.ai/v1",  # n4n.ai OpenAI-compatible endpoint
    api_key=os.environ["N4N_KEY"],
)
resp = n4n.chat.completions.create(
    model="meta-llama/llama-3.1-8b-instruct",  # unified naming
    messages=[{"role": "user", "content": "ping"}],
    stream=False,
)

Both support Server-Sent Events streaming, JSON response formats, and tool calls. The request schema is identical, but n4n normalizes model names across providers, so you lose the accounts/fireworks/models/ prefix and gain portability to other backends without rewriting your client.

Streaming differences

Fireworks streams tokens with standard SSE data: frames. n4n does the same, but because it may route to a different provider mid-request during fallback, the model field in the final usage object can differ from the requested one. Your logging must read the response, not assume the request string.

Model coverage and routing

Fireworks is a single-vendor inference service: you get their hosted open-weight models, fine-tunes, and some partnered weights. n4n is a gateway in front of 240+ models from many providers. When you send a request to n4n, you can pin a specific provider or let the gateway route.

{
  "model": "anthropic/claude-3.5-sonnet",
  "messages": [{"role": "user", "content": "summarize"}],
  "route": {"prefer": ["fireworks", "together"]}
}

That route directive is honored by n4n; Fireworks has no equivalent because it only serves its own fleet. If your stack hard-codes Fireworks model IDs, you will rewrite those strings and possibly your model-selection logic. Fireworks also exposes version-pinned model hashes (e.g., @20240820); n4n abstracts those behind provider/model tags, which is convenient but means you lose exact snapshot control unless you specify a provider that supports it.

Cost model and metering

Fireworks bills per token at rates published on their pricing page, with separate lines for input, output, and sometimes cached tokens. You get a monthly invoice and usage dashboard per API key.

n4n applies per-token usage metering across all routed providers, meaning a single bill line aggregates Claude, Llama, and Mixtral usage. The gateway does not mark up tokens arbitrarily—it passes through provider cost plus a gateway fee (check current terms). The practical difference: with Fireworks you optimize within one vendor; with n4n you can shift traffic to cheaper models mid-month without code changes.

# Fireworks usage is scoped to fireworks models only
curl https://api.fireworks.ai/usage/v1/...

# n4n returns unified meter across providers
curl https://api.n4n.ai/v1/usage | jq '.tokens_by_model'

Fireworks offers a batch inference API with discounted rates for offline jobs. n4n does not currently aggregate batch discounts across providers; if you rely on 50% off batch pricing, you keep that workload on Fireworks or negotiate directly with the upstream.

Latency and throughput characteristics

Fireworks built its reputation on low-latency decoding for open models, often using custom CUDA kernels. If you benchmark a 8B model on Fireworks, you see tight time-to-first-token (TTFT) because the request never leaves their network.

n4n adds a routing hop. For a model hosted by Fireworks but accessed via n4n, you pay a small proxy penalty. For models hosted elsewhere, latency is dominated by the upstream provider. The gateway does keep connections warm and forwards cache-control hints, so repeated prompts with cache_control can hit provider prompt caches as if you called them directly.

Do not assume n4n is slower across the board; if your traffic spans multiple vendors, consolidating on one gateway reduces TLS handshakes and client connection pool fragmentation. Measure with a representative payload:

import time, statistics
def ttft(client, model, prompt):
    start = time.monotonic()
    stream = client.chat.completions.create(
        model=model, messages=[{"role":"user","content":prompt}], stream=True)
    for chunk in stream:
        if chunk.choices[0].delta.content:
            return time.monotonic() - start

Run that against both bases with the same model weight to quantify the proxy delta for your region.

Failure handling and fallback

This is where switching from Fireworks AI to n4n changes operational posture most. Fireworks returns 429 or 503 when its fleet is saturated; your code must implement backoff or fail.

n4n provides automatic fallback when a provider is rate-limited or degraded. If you request meta-llama/llama-3.1-70b and the primary provider is throwing 429, the gateway retries another provider hosting the same weight, assuming one is available. You can disable this with a header.

POST /v1/chat/completions
x-n4n-fallback: false

That single feature removes a class of retry code from your services. On Fireworks, you wrote that retry loop yourself. Note that fallback may change the response latency profile and the exact model version served; for strict reproducibility, disable it and pin a route.

Ecosystem and tooling

Fireworks offers a Python SDK, fine-tuning console, and batch inference UI. Their differentiator is model customization: upload data, train a LoRA, deploy with one click. n4n is deliberately provider-agnostic. It does not host fine-tunes; it routes to wherever the model lives. Its ecosystem is the OpenAI client ecosystem—anything that speaks /v1/chat/completions works.

If you rely on Fireworks-specific features like fireworks-tuned models or their eval harness, those do not exist on n4n. You keep the generic tooling and lose vendor lock-in. For teams already using LangChain or Haystack, both endpoints are interchangeable behind the OpenAI chat model wrapper.

Limits and quotas

Fireworks enforces per-key RPM/TPM limits documented in their console. n4n inherits the limits of upstream providers and layers its own gateway quotas. A request to a model behind a provider with a 10k TPM cap will hit that cap even through n4n. The gateway surfaces the upstream retry-after header rather than masking it.

One subtle limit: Fireworks allows long context windows on specific models (e.g., 128k). n4n passes context length through, but if you route to a provider that truncates, the gateway will not pad it. Your prompt validation must become provider-aware or you pin a known-good route.

Migration code example

A minimal shift for a chat proxy:

import os
from openai import OpenAI

def get_client(use_fw: bool) -> OpenAI:
    if use_fw:
        return OpenAI(
            base_url="https://api.fireworks.ai/inference/v1",
            api_key=os.environ["FW_KEY"],
        )
    return OpenAI(
        base_url="https://api.n4n.ai/v1",  # n4n.ai endpoint
        api_key=os.environ["N4N_KEY"],
    )

def complete(prompt: str, model: str, use_fw: bool = False):
    client = get_client(use_fw)
    # model name must be mapped when not using FW
    mapped = model.replace("accounts/fireworks/models/", "")
    return client.chat.completions.create(
        model=mapped,
        messages=[{"role": "user", "content": prompt}],
        temperature=0.2,
    )

The mapping step is the only mandatory logic change; everything else is configuration. For a phased rollout, run both clients behind a flag and compare outputs on a golden set.

Head-to-head summary

Which to choose

Stay on Fireworks if: You depend on Fireworks-native fine-tunes, need the absolute lowest TTFT on a specific open model, and have no desire to multi-home. The single-vendor SLA is simpler to reason about, and batch discounts are valuable for offline jobs.

Switch to n4n if: You are building a product that must survive provider outages, want to mix Claude, Llama, and Mixtral without negotiating five contracts, or need unified metering. Switching from Fireworks AI to n4n is also the right call when your traffic shape changes monthly and you want to shift models by config, not by code deploy.

Hybrid: Keep Fireworks for steady-state fine-tuned inference, front experimental traffic through n4n to exploit fallback. Both speak the same client interface, so a flag flips between them.

The migration is small in lines of code and large in operational behavior. Plan for the routing and fallback differences, and the rest is string substitution.