LLM API pagination list endpoints are REST collection routes that return subsets of a larger dataset across multiple requests instead of one giant response. They use offset, limit, or cursor parameters so clients can iterate through models, files, or job records without overloading memory or network buffers.
What the pattern actually covers
In LLM platforms, “list” endpoints appear wherever the system accumulates records: /v1/models, /v1/files, /v1/fine_tuning/jobs, /v1/batches. Each returns a JSON object containing an array plus pagination metadata. The alternative—returning every record in one call—breaks at scale. A gateway aggregating provider catalogs can expose hundreds of models; a single response with full metadata would blow past proxy timeouts.
Pagination is not a nicety. It is the contract that keeps list calls bounded in latency and memory.
Offset and limit: the naive but useful baseline
The simplest llm api pagination list endpoints use two query parameters: limit (page size) and offset (skip count). The server slices the result set like SQL LIMIT offset, limit.
curl "https://api.example.com/v1/files?limit=10&offset=20"Response:
{
"object": "list",
"data": [ { "id": "file-abc", "bytes": 1234 }, ... ],
"has_more": true
}Client loops:
import requests
base = "https://api.example.com/v1/files"
offset = 0
while True:
r = requests.get(base, params={"limit": 100, "offset": offset})
r.raise_for_status()
page = r.json()
for item in page["data"]:
process(item)
if not page.get("has_more"):
break
offset += len(page["data"])Offset pagination is easy to reason about. It also lies: if items are inserted or deleted between calls, you skip or duplicate rows. For a static model catalog that changes weekly, that is acceptable. For live job queues, it is a bug factory.
Cursor-based pagination: the production default
Cursor (or keyset) pagination replaces offset with an opaque token representing a position in the ordered set. The server returns next_cursor instead of has_more alone.
curl "https://api.example.com/v1/fine_tuning/jobs?limit=20&after=ftjob_xyz"{
"object": "list", "data": [ { "id": "ftjob-111" }, ... ],
"next_cursor": "ftjob-999"
}Iterate:
cursor = None
while True:
params = {"limit": 20}
if cursor:
params["after"] = cursor
r = requests.get("https://api.example.com/v1/fine_tuning/jobs", params=params)
r.raise_for_status()
page = r.json()
for job in page["data"]:
monitor(job)
cursor = page.get("next_cursor")
if not cursor:
breakThe token encodes the last seen sort key (often creation timestamp + id). New jobs inserted after your start point do not shift your window. This matters when you are auditing every completion job across a week.
Why this matters for LLM gateways
A unified gateway such as n4n.ai exposes an OpenAI-compatible /v1/models endpoint that paginates across 240+ models behind one interface; the client sends the same cursor logic regardless of whether the underlying provider is OpenAI, Anthropic, or a self-hosted vLLM. Without pagination, a single model list call would serialize provider responses and risk exceeding the gateway’s own upstream timeouts.
Even if you only call one provider, their catalog grows. Cursor pagination lets you lazily load the subset you need—say, only models whose id matches gpt-4*—by filtering client-side after fetching pages.
Concrete example: paging a model catalog in TypeScript
Assume we want to collect all model ids from a gateway to build a local cache.
interface ModelList {
object: "list";
data: { id: string; created: number }[];
next_cursor?: string;
}
async function getAllModelIds(baseUrl: string): Promise<string[]> {
const ids: string[] = [];
let cursor: string | undefined;
do {
const url = new URL(`${baseUrl}/v1/models`);
url.searchParams.set("limit", "50");
if (cursor) url.searchParams.set("after", cursor);
const res = await fetch(url.toString());
if (!res.ok) throw new Error(`list failed: ${res.status}`);
const page = (await res.json()) as ModelList;
for (const m of page.data) ids.push(m.id);
cursor = page.next_cursor;
} while (cursor);
return ids;
}This pattern never holds more than one page in memory. It also degrades gracefully: if the process crashes at page 4, you can restart from the last persisted next_cursor.
Common misconceptions
“Pagination is just a database concern”
False. LLM list endpoints often aggregate from multiple upstream APIs. The gateway must merge and stable-sort before paginating. The cursor may span heterogeneous sources.
“Offset is fine because lists are small”
At small scale, yes. But model catalogs and file stores grow silently. The day your offset=10000 query triggers a sequential scan on the provider side, your p99 latency spikes. Cursor avoids deep offsets entirely.
“A boolean has_more is enough”
has_more: true tells you to continue, but not where. You still need an explicit offset or implicit cursor. Many APIs omit has_more and use empty next_cursor to signal termination—cleaner, because it couples the continuation token with the signal.
“REST pagination uses Link headers”
Some APIs do (GitHub). Most LLM vendors embed pagination in the JSON body because it survives JSON-only SDKs and websocket proxies. Do not assume RFC 5988 Link headers exist; read the schema.
“List order is stable by id”
Providers sort by creation time, not id. UUIDs are not monotonic. If you sort client-side, you may interleave pages incorrectly. Trust the server’s order and append.
Designing your own list endpoint
If you build a gateway or internal LLM tool, follow these rules:
- Support
limit(max 100, default 20). Reject absurd values. - Return
next_cursor(opaque) notoffset. Encode{sort_key, id}encrypted or hashed. - Include
object: "list"anddataarray to match OpenAI shape. - Document that cursors expire after some TTL; clients should not store them for days.
- Never return total count unless cheap;
SELECT COUNT(*)on sharded metadata is a latency trap.
Example response shape:
{
"object": "list",
"data": [ { "id": "model-1" } ],
"next_cursor": "eyJjcmVhdGVkIjoxNzA..."
}Client-side accumulation patterns
For batch sync, write pages to disk as you fetch:
import json, requests
with open("models.jsonl", "w") as f:
cursor = None
while True:
p = {"limit": 50}
if cursor: p["after"] = cursor
page = requests.get("https://api.example.com/v1/models", params=p).json()
for m in page["data"]:
f.write(json.dumps(m) + "\n")
cursor = page.get("next_cursor")
if not cursor:
breakFor interactive UIs, fetch one page per scroll event. Do not pre-fetch all pages; the user may never scroll.
Edge cases that bite
- Empty first page:
data: []andnext_cursor: null. Your loop must handle zero-length pages without infinite retry. - Cursor invalidation: If the underlying data reshuffles, the server may return 400 on an old cursor. Exponential backoff and restart from scratch.
- Limit clamping: Server silently reduces limit to max. Client should read actual page length, not assume requested limit.
Summary of the contract
llm api pagination list endpoints exist to make unbounded collections tractable over stateless HTTP. Use cursor pagination when data changes or scales; use offset only for static, small sets. Encode the continuation token in the response body, iterate until it is absent, and never assume global order beyond what the server returns.