For high-volume workloads
Same call. A little patience. A better rate.
Instant batch inference isn't a separate endpoint, a job object, or a file upload — it's two parameters,ibi andqueue_factor, on the normal /v1/chat/completions call you're already making. Opt in and n4n holds your request in a shared queue with other opted-in traffic for that model, dispatching it upstream as a batch once the queue's ready — you still get back one response through the same call, just a little later, at a lower rate and without competing 1:1 for the provider's per-key rate limit.
Instant batch inference — live queue
Illustrative — a simulated queue, not live telemetry. queue_factor trades latency for a deeper discount.
How it works
One extra parameter. Same request shape.
01
Flip on ibi
Add "ibi": true to any normal chat completions call — same endpoint, same single request and response.
02
n4n pools the queue
Your request joins a shared queue with other ibi traffic for that model. queue_factor (0–1) sets how long it waits before the queue dispatches.
03
Get a discounted response
Once the batch dispatches upstream, your response comes back through the same call — priced at the batch rate.
Illustrative — request shape, subject to change
curl https://api.n4n.ai/v1/chat/completions \ -H "Authorization: Bearer n4n_your_key" \ -H "Content-Type: application/json" \ -d '{"model":"anthropic/claude-sonnet-5","messages":[{"role":"user","content":"Hello"}],"ibi":true,"queue_factor":0.5}'
Higher queue_factor trades more latency for a deeper discount and more rate-limit headroom; lower values dispatch sooner for a smaller discount. Requests withoutibi are unaffected — same routing and pricing as always.
Which one do I want?
Instant batch inference vs. load factor.
Instant batch inference
Any model, any request volume. Same call, a little delay — trade latency for a lower rate and rate-limit headroom.
Load factor pricing
Open-weights models only, no opt-in. Requests bill at list price and the discount — up to 90%, depending on your plan — is credited back after the hour closes.
See how it works →Try instant batch inference.
Sign in, create a key, and add ibi to your next request in under a minute.