Calling LLM endpoints from a shell script is deceptively easy until a provider returns a 429 or drops the TCP connection mid-stream. A solid bash retry function llm api calls saves your batch jobs from failing on the first transient error, and it is simpler to implement than most engineers expect. You can build one in under 50 lines that handles backoff, jitter, and non-retryable status codes correctly.
Step 1: Decide what qualifies as a retryable failure
Not every non-zero exit code should trigger a retry. A 401 means your API key is wrong; hammering the endpoint will not fix it. A 429 or 503 usually means temporary capacity limits. Network errors from curl (exit code 7, 28, 56) are also retryable.
Start by writing a predicate that inspects an HTTP status code and a curl exit code:
is_retryable() {
local http_code="$1"
local curl_exit="$2"
# Retry on 429, 5xx, and curl transport errors
if [ "$curl_exit" -ne 0 ]; then
case "$curl_exit" in
7|28|35|56|92|93) return 0 ;; # conn refused, timeout, ssl, recv, etc.
*) return 1 ;;
esac
fi
if [ -z "$http_code" ]; then return 1; fi
if [ "$http_code" -eq 429 ] || [ "$http_code" -ge 500 ]; then
return 0
fi
return 1
}This keeps the retry logic honest: auth and validation errors fail fast.
Step 2: Write the core bash retry function llm api loop
The skeleton is a while loop with an attempt counter. We capture both the HTTP status and the curl exit code using -w and -o.
retry_llm() {
local max_attempts="${1:-5}"
local attempt=1
local http_code curl_exit
local resp_file
resp_file=$(mktemp)
while [ "$attempt" -le "$max_attempts" ]; do
curl_exit=0
http_code=$(curl -sS -o "$resp_file" -w '%{http_code}' \
"$@" ) || curl_exit=$?
if ! is_retryable "$http_code" "$curl_exit"; then
cat "$resp_file"
rm -f "$resp_file"
return "$curl_exit"
fi
echo "Attempt $attempt failed (http=$http_code curl=$curl_exit). Retrying..." >&2
attempt=$((attempt + 1))
sleep 1 # placeholder, replaced with backoff in Step 3
done
echo "All $max_attempts attempts failed" >&2
cat "$resp_file"
rm -f "$resp_file"
return 1
}The "$@" passes through all curl arguments, so the caller supplies URL, headers, and body. This bash retry function llm api wrapper stays generic.
Step 3: Add exponential backoff with jitter
Fixed 1-second sleeps are either too aggressive or too slow. Use exponential growth capped at, say, 30 seconds, plus random jitter to avoid thundering herds.
backoff_sleep() {
local attempt="$1"
local base=2
local max=30
local exp=$(( base ** (attempt - 1) ))
[ "$exp" -gt "$max" ] && exp=$max
# jitter: random float between 0 and 1 via awk
local jitter
jitter=$(awk -v seed="$RANDOM" 'BEGIN{srand(seed); print rand()}')
local total
total=$(awk -v e="$exp" -v j="$jitter" 'BEGIN{printf "%.2f", e * j + e * 0.5}')
sleep "$total"
}Replace the sleep 1 in Step 2 with backoff_sleep "$attempt". The delay scales 1s → 2s → 4s → 8s, with ±50% noise.
Step 4: Wrap a real OpenAI-compatible chat completion call
Most LLM gateways speak the OpenAI chat completions shape. Here is a concrete invocation using retry_llm:
call_llm() {
local prompt="$1"
retry_llm 5 \
-X POST "https://api.openai.com/v1/chat/completions" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d "$(jq -n --arg p "$prompt" '{
model: "gpt-4o-mini",
messages: [{role: "user", content: $p}],
temperature: 0.2
}')"
}If you front requests with n4n.ai, the gateway already performs automatic fallback when a provider is rate-limited or degraded, so client-side retries can be thinner. For direct provider access, the bash retry function llm api above is essential.
Step 5: Fail fast on non-retryable JSON errors
Some providers return 200 with an embedded error, or 400 with a JSON body. Parse the response to catch auth failures even when curl succeeded:
check_json_error() {
# expects response on stdin
local body
body=$(cat)
local err
err=$(echo "$body" | jq -r '.error // empty')
if [ -n "$err" ]; then
echo "Non-retryable API error: $err" >&2
return 1
fi
echo "$body"
return 0
}Pipe call_llm output through this before consuming the result:
response=$(call_llm "Summarize: $input" | check_json_error) || exit 1
echo "$response" | jq -r '.choices[0].message.content'Step 6: Test the bash retry function llm api against a flaky mock
Do not test against a paid endpoint. Stand up a mock that fails the first two calls, then succeeds:
mock_server() {
local count=0
while true; do
read -r req
count=$((count + 1))
if [ "$count" -le 2 ]; then
printf 'HTTP/1.1 503 Service Unavailable\r\nContent-Length: 0\r\n\r\n'
else
printf 'HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n\r\n{"ok":true}'
fi
done | nc -l 8080
}In another shell, point retry_llm at http://localhost:8080 and confirm it retries twice then prints {"ok":true}. Adjust max_attempts to 3 for the test.
A simpler pure-bash test uses a function that simulates curl:
fake_curl() {
local fails_left_file="$1"
local left
left=$(cat "$fails_left_file")
if [ "$left" -gt 0 ]; then
echo $((left - 1)) > "$fails_left_file"
echo "503" # http_code
return 7 # curl exit
fi
echo "200"
return 0
}Drive your loop with this to verify attempt counting and backoff without network dependencies.
Step 7: Verify success and operationalize
Success means the script exits 0 and emits valid JSON on the final attempt. Add a wrapper that logs attempt metadata:
export LLM_LOG="${LLM_LOG:-/var/log/llm_retry.log}"
log_attempt() {
echo "$(date -u +%FT%TZ) $*" >> "$LLM_LOG"
}Call log_attempt inside the retry loop. In production, alert if attempt exceeds 3 within a short window—that signals a provider outage rather than a blip.
For per-token metering or routing directives, a gateway can forward cache-control hints and handle fallback, but your shell job still owns timeouts and DNS issues. The bash retry function llm api we built covers the client side completely.
Verify by running a real batch of 100 prompts through call_llm with a deliberately low rate limit, then check that zero jobs died and the log shows expected backoff sleeps. Tune base and max to your provider’s retry-After headers if they send them.