How to call OpenAI-compatible models from Django views

Most Django apps need a clean seam between request handling and external model calls. This guide shows how to call django views openai-compatible models without tangling your HTTP layer with SDK specifics. We’ll stand up a minimal view that proxies a chat completion request to any OpenAI-compatible endpoint, then harden it for production.

Step 1: Install dependencies and configure the client

You need the official OpenAI Python package. It speaks the OpenAI-compatible contract, so pointing it at another base URL works without code changes.

pip install openai django

Create a small module to hold the client. Keep credentials in environment variables, not in settings.py committed to git.

# llm_client.py
import os
from openai import OpenAI

def get_client():
    return OpenAI(
        api_key=os.environ["LLM_API_KEY"],
        base_url=os.environ.get("LLM_BASE_URL", "https://api.openai.com/v1"),
    )

If you want access to 240+ models behind one contract, point LLM_BASE_URL at a gateway such as n4n.ai. It exposes a single OpenAI-compatible endpoint and automatically falls back when a provider is rate-limited or degraded.

Step 2: Write a minimal Django view

A function-based view keeps the example readable. It reads a prompt from JSON, calls the model, and returns the completion.

# views.py
import json
from django.http import JsonResponse
from llm_client import get_client

def chat_view(request):
    if request.method != "POST":
        return JsonResponse({"error": "POST required"}, status=405)
    try:
        body = json.loads(request.body)
        prompt = body["prompt"]
    except (json.JSONDecodeError, KeyError):
        return JsonResponse({"error": "invalid json, need prompt"}, status=400)

    client = get_client()
    resp = client.chat.completions.create(
        model=body.get("model", "gpt-4o-mini"),
        messages=[{"role": "user", "content": prompt}],
        temperature=0.7,
    )
    return JsonResponse({"content": resp.choices[0].message.content})

This is the core pattern for django views openai-compatible models: treat the LLM as a remote service, not a library import.

Step 3: Validate inbound requests

Never trust raw JSON in production. Use Django’s forms or a lightweight pydantic model. Below is a form-based guard.

# forms.py
from django import forms

class ChatForm(forms.Form):
    prompt = forms.CharField(min_length=1, max_length=4000)
    model = forms.CharField(required=False, max_length=100)

Wire it into the view:

from .forms import ChatForm

def chat_view(request):
    if request.method != "POST":
        return JsonResponse({"error": "POST required"}, status=405)
    form = ChatForm(json.loads(request.body) if request.body else None)
    if not form.is_valid():
        return JsonResponse({"error": form.errors}, status=400)
    prompt = form.cleaned_data["prompt"]
    model = form.cleaned_data.get("model") or "gpt-4o-mini"
    # ... call client as before

Step 4: Stream tokens back to the browser

Users expect incremental output. OpenAI-compatible APIs support stream=True. Django’s StreamingHttpResponse handles it.

import json
from django.http import StreamingHttpResponse
from llm_client import get_client

def stream_chat_view(request):
    if request.method != "POST":
        return JsonResponse({"error": "POST required"}, status=405)
    body = json.loads(request.body)
    prompt = body["prompt"]
    model = body.get("model", "gpt-4o-mini")

    def event_stream():
        client = get_client()
        stream = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            stream=True,
        )
        for chunk in stream:
            if chunk.choices[0].delta.content:
                yield "data: " + json.dumps({"token": chunk.choices[0].delta.content}) + "\n\n"

    return StreamingHttpResponse(event_stream(), content_type="text/event-stream")

The pattern for django views openai-compatible models stays identical whether you buffer or stream; only the response wrapper changes.

Step 5: Timeouts, retries, and error mapping

Network calls fail. Set a timeout on the client and map exceptions to HTTP status codes.

from openai import APIConnectionError, APIStatusError, APITimeoutError

def chat_view(request):
    # ... validation ...
    client = get_client()
    try:
        resp = client.chat.completions.create(
            model=model,
            messages=[{"role": "user", "content": prompt}],
            timeout=15,
        )
    except APITimeoutError:
        return JsonResponse({"error": "upstream timeout"}, status=504)
    except APIConnectionError:
        return JsonResponse({"error": "upstream unreachable"}, status=502)
    except APIStatusError as e:
        return JsonResponse({"error": f"upstream error {e.status_code}"}, status=502)
    return JsonResponse({"content": resp.choices[0].message.content})

If you route through a gateway, automatic fallback reduces the frequency of these errors, but your view must still handle the residual cases.

Step 6: Verify the integration end to end

Start the dev server and hit the view with curl.

./manage.py runserver
curl -X POST http://127.0.0.1:8000/chat/ \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Say hello in three words.","model":"gpt-4o-mini"}'

Expected success response:

{"content":"Hello there, friend."}

For the streaming endpoint:

curl -N -X POST http://127.0.0.1:8000/chat/stream/ \
  -H "Content-Type: application/json" \
  -d '{"prompt":"Count to three."}'

You should see SSE frames arrive one per token. If you routed through n4n.ai, the per-token usage metering appears in the response headers, letting you confirm billing without extra instrumentation.

Add a Django test to lock the behavior:

from django.test import TestCase, Client
from unittest.mock import patch

class ChatViewTest(TestCase):
    @patch("llm_client.get_client")
    def test_chat_returns_content(self, mock_get):
        mock_get.return_value.chat.completions.create.return_value.choices = [
            type("C", (), {"message": type("M", (), {"content": "hi"})()})()
        ]
        c = Client()
        resp = c.post("/chat/", data='{"prompt":"hi"}', content_type="application/json")
        self.assertEqual(resp.status_code, 200)
        self.assertEqual(resp.json()["content"], "hi")

That test isolates your django views openai-compatible models code from network flakiness.

Production notes

Run sync views behind a worker pool with limited concurrency; LLM calls are I/O bound but can hold connections. For higher throughput, wrap the client in an async view using httpx and openai.AsyncOpenAI, but keep the same URL and message shape.

Cache completions for identical prompts when the model is deterministic. Honor provider cache-control hints if your gateway forwards them; this avoids redundant spend.

Logging should capture model name, token count, and latency. The OpenAI response object includes usage fields; emit them to your metrics pipeline.

Following these steps gives you a maintainable integration that swaps providers by changing one environment variable. The contract is stable, the code is boring, and that is exactly what you want.