Calling an external model from a web request is a quick way to blow up your p95. Using laravel queue jobs llm api integration lets you absorb latency spikes, retry transient provider errors, and keep user-facing routes fast. This guide walks through a production-shaped setup, not a toy snippet.
Step 1: Configure a queue driver
Start with the database driver if you have no Redis yet. It needs zero extra infrastructure and survives restarts.
Set the connection in .env:
QUEUE_CONNECTION=databaseThen create the jobs and failed_jobs tables:
php artisan queue:table
php artisan queue:failed-table
php artisan migrateFor higher throughput, switch to Redis later by setting QUEUE_CONNECTION=redis and running php artisan config:cache. The job code does not change. Do not ship the sync driver outside local development—it executes inline and defeats the entire purpose.
Step 2: Generate the LLM job class
Create a dedicated job for the API call. Keeping the HTTP logic inside a job makes it retryable and observable.
php artisan make:job ProcessLLMCompletionFirst, define the service config so credentials never touch the serialized job payload:
// config/services.php
'llm' => [
'endpoint' => env('LLM_ENDPOINT', 'https://api.openai.com/v1/chat/completions'),
'key' => env('LLM_API_KEY'),
],A minimal, correct implementation that hits an OpenAI-compatible endpoint:
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;
class ProcessLLMCompletion implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public int $tries = 5;
public int $backoff = 10;
public function __construct(
private string $prompt,
private string $correlationId
) {}
public function handle(): void
{
$response = Http::withToken(config('services.llm.key'))
->timeout(30)
->post(config('services.llm.endpoint'), [
'model' => 'gpt-4o-mini',
'messages' => [
['role' => 'user', 'content' => $this->prompt],
],
'temperature' => 0.2,
]);
if ($response->failed()) {
// Laravel will retry based on $tries/$backoff
$response->throw();
}
$content = $response->json('choices.0.message.content');
Log::info('LLM done', [
'correlation_id' => $this->correlationId,
'length' => strlen($content),
]);
// Persist or dispatch downstream event here
event(new \App\Events\LLMCompleted($this->correlationId, $content));
}
}The throw() call on a failed response triggers Laravel’s retry machinery. A 429 or 503 from the provider releases the job back to the queue after the backoff.
Step 3: Dispatch the job from your app
Never call the LLM synchronously inside a controller. Dispatch and return a 202 with a polling token.
<?php
namespace App\Http\Controllers;
use App\Jobs\ProcessLLMCompletion;
use Illuminate\Http\Request;
use Illuminate\Support\Str;
class PromptController extends Controller
{
public function store(Request $request)
{
$correlationId = Str::uuid()->toString();
$prompt = $request->input('prompt');
ProcessLLMCompletion::dispatch($prompt, $correlationId)
->onQueue('llm');
return response()->json(['id' => $correlationId], 202);
}
}The onQueue('llm') call isolates model traffic from email or image jobs so one slow provider does not stall your password-reset mailer.
Step 4: Tune retries for provider reality
LLM endpoints fail in ways conventional APIs do not: rate limits are bursty, and generation can time out at 30s+. Your laravel queue jobs llm api layer should assume at least one retry per hundred calls.
Set explicit backoff. Exponential is overrated; fixed 10–15s works because most provider throttles reset on a sliding window.
public int $tries = 6;
public array $backoff = [5, 10, 20, 40, 80];If you point at an OpenAI-compatible gateway such as n4n.ai, its automatic fallback when a provider is rate-limited or degraded means a single 5xx from one backend may already be resolved upstream—your job still retries, but the second attempt often lands on a healthy path without extra code.
Step 5: Make jobs failure-safe and idempotent
A job that fires twice must not double-charge a customer or post two support replies. Use the correlation ID as a unique key in your datastore.
Add a failed() hook to alert:
public function failed(\Throwable $exception): void
{
Log::error('LLM job permanently failed', [
'correlation_id' => $this->correlationId,
'error' => $exception->getMessage(),
]);
// Optionally notify Sentry or PagerDuty
}For strict idempotency, check the cache before handling:
if (Cache::has("llm:{$this->correlationId}")) {
return;
}
Cache::put("llm:{$this->correlationId}", true, now()->addHour());Do this at the top of handle(). Use a Redis or memcached store for the lock so it survives worker restarts.
Step 6: Run the worker and verify success
Start a worker scoped to the llm queue:
php artisan queue:work --queue=llm --tries=6Verification checklist:
- Dispatch a request via
curlor the controller route. - Confirm the
jobstable row count returns to zero within ~30s. - Check
storage/logs/laravel.logfor theLLM doneinfo line with your correlation ID. - If you persisted the result, query the table for that ID.
A quick smoke test from the command line:
curl -X POST https://app.test/api/prompt \
-H "Content-Type: application/json" \
-d '{"prompt":"Summarize queuing theory in one sentence"}'Then:
php artisan tinker
>>> DB::table('llm_results')->where('correlation_id', 'UUID-FROM-RESPONSE')->exists();If it returns true, the laravel queue jobs llm api pipeline works end to end. If the row never appears, check the failed_jobs table before blaming the network.
Step 7: Batch multiple prompts
When you need ten summaries at once, use job batches instead of firing ten disconnected jobs.
use Illuminate\Support\Facades\Bus;
use App\Jobs\ProcessLLMCompletion;
$batch = Bus::batch([])->then(function () {
Log::info('All LLM jobs finished');
})->dispatch();
foreach ($prompts as $p) {
$batch->add(new ProcessLLMCompletion($p, Str::uuid()->toString()));
}Batches give you a single completion callback and a central failure point. They also let you cap concurrency with --queue=llm --max-jobs=4 so you stay under provider RPM limits.
Step 8: Monitor and cap throughput
Queue workers hide backpressure. If the llm queue depth grows, you are exceeding your provider quota. Use Laravel’s queue:monitor or a Prometheus exporter on the jobs table.
Set a sensible --timeout=60 on the worker so a stuck HTTP call does not pin a process forever. Pair it with the HTTP timeout(30) in the job. Never let the worker timeout be lower than the HTTP timeout plus serialization overhead.
Step 9: Test the job in isolation
Use Laravel’s fake HTTP and queue fakes to guard against regressions when you swap models or endpoints.
public function test_job_calls_api_and_dispatches_event()
{
Http::fake([
'*' => Http::response([
'choices' => [['message' => ['content' => 'ok']]]
], 200),
]);
Event::fake();
$job = new ProcessLLMCompletion('test', 'id-1');
$job->handle();
Event::assertDispatched(\App\Events\LLMCompleted::class);
}This runs in milliseconds and proves your request shape matches the provider schema.
Production notes
Do not put your API key in the job payload. It is serialized to the queue; use config() inside handle(). Mark the job SerializesModels only if passing Eloquent models—here we pass scalars, which is cheaper and avoids lazy-loading surprises.
If you later adopt per-token usage metering, read the usage field from the response and write it to a metrics table inside the same job transaction. That keeps cost attribution exactly once per successful generation.
The pattern above keeps your HTTP routes at single-digit millisecond response times while the heavy generation work churns through a durable queue. That is the only way to run laravel queue jobs llm api calls without apologizing for latency.