Adding an LLM chat feature to a Django app

Most Django apps that add a chat feature end up doing the same five things: storing conversation history, calling a chat completions API, streaming the response back to the browser, handling provider failures gracefully, and keeping token costs in check. This tutorial builds all five, using Django’s ORM for storage, an OpenAI-compatible chat completions endpoint for inference, and htmx for streaming without a separate frontend build step. The code works with any OpenAI-compatible provider — swap the base URL and key if you’re using something else.

Step 1: model the conversation

A chat feature needs at least two tables: conversations and messages. Keep it simple — you can add titles, user ownership, and soft deletes later.

# chat/models.py
import uuid
from django.conf import settings
from django.db import models


class Conversation(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
    title = models.CharField(max_length=200, blank=True)
    created_at = models.DateTimeField(auto_now_add=True)

    class Meta:
        ordering = ["-created_at"]


class Message(models.Model):
    ROLE_CHOICES = [("user", "user"), ("assistant", "assistant"), ("system", "system")]

    conversation = models.ForeignKey(Conversation, on_delete=models.CASCADE, related_name="messages")
    role = models.CharField(max_length=16, choices=ROLE_CHOICES)
    content = models.TextField()
    created_at = models.DateTimeField(auto_now_add=True)

    class Meta:
        ordering = ["created_at"]

Run python manage.py makemigrations chat && python manage.py migrate before moving on.

Step 2: install a client and set config

You don’t need a heavyweight SDK — any OpenAI-compatible endpoint works with the official openai Python package pointed at a different base_url, or with plain requests/httpx. We’ll use openai here because its streaming interface is convenient.

pip install openai

Add the connection settings to settings.py, reading from the environment so the key never lands in source control:

# settings.py
import os

LLM_API_BASE = os.environ.get("LLM_API_BASE", "https://api.n4n.ai/v1")
LLM_API_KEY = os.environ["LLM_API_KEY"]
LLM_MODEL = os.environ.get("LLM_MODEL", "openai/gpt-4o-mini")

Pointing LLM_API_BASE at a gateway rather than a single provider’s API is worth doing even in a tutorial-sized project: n4n.ai fronts 240+ models from many providers behind one OpenAI-compatible endpoint, so LLM_MODEL becomes a config value instead of a vendor lock-in decision, and pay-per-token billing means you’re not committing to a monthly seat just to prototype a chat feature.

Step 3: write a thin service layer

Keep the API call out of your views so it’s testable and reusable from a management command or Celery task later.

# chat/services.py
from django.conf import settings
from openai import OpenAI

client = OpenAI(base_url=settings.LLM_API_BASE, api_key=settings.LLM_API_KEY)


def build_messages(conversation):
    history = conversation.messages.values("role", "content")
    return [{"role": m["role"], "content": m["content"]} for m in history]


def stream_reply(conversation):
    """Yields content chunks as they arrive from the model."""
    messages = build_messages(conversation)
    response = client.chat.completions.create(
        model=settings.LLM_MODEL,
        messages=messages,
        stream=True,
        max_tokens=1024,
    )
    for chunk in response:
        delta = chunk.choices[0].delta.content
        if delta:
            yield delta

build_messages sends the full conversation on every turn, which is the correct default for short conversations. Once histories get long enough to blow the context window, truncate to the last N messages or summarize older turns — but don’t add that complexity before you need it.

Step 4: the view — a synchronous streaming endpoint

Django can stream a response without going async, using StreamingHttpResponse. This is the simplest reliable option and works fine behind gunicorn or uWSGI with a threaded worker.

# chat/views.py
from django.http import StreamingHttpResponse
from django.shortcuts import get_object_or_404
from django.views.decorators.http import require_POST
from django.contrib.auth.decorators import login_required

from .models import Conversation, Message
from .services import stream_reply


@login_required
@require_POST
def send_message(request, conversation_id):
    conversation = get_object_or_404(Conversation, id=conversation_id, user=request.user)
    user_text = request.POST.get("content", "").strip()
    if not user_text:
        return StreamingHttpResponse(iter([""]), status=400)

    Message.objects.create(conversation=conversation, role="user", content=user_text)

    def event_stream():
        chunks = []
        try:
            for delta in stream_reply(conversation):
                chunks.append(delta)
                yield delta
        finally:
            full_text = "".join(chunks)
            if full_text:
                Message.objects.create(conversation=conversation, role="assistant", content=full_text)

    response = StreamingHttpResponse(event_stream(), content_type="text/plain")
    response["Cache-Control"] = "no-cache"
    response["X-Accel-Buffering"] = "no"  # disable nginx buffering for real streaming
    return response

The finally block matters: even if the client disconnects mid-stream, you still persist whatever the model generated up to that point, so the conversation history stays accurate for the next turn.

X-Accel-Buffering: no is easy to forget and is the most common reason a “streaming” endpoint arrives as one big chunk in production — nginx buffers proxied responses by default.

Step 5: wire up the frontend with htmx

You don’t need a JS framework for a chat box. htmx’s sse extension or a plain fetch-and-read-stream loop both work; here’s a minimal fetch-based version that appends tokens as they arrive.

<!-- templates/chat/conversation.html -->
<div id="messages">
  {% for message in conversation.messages.all %}
    <div class="msg msg-{{ message.role }}">{{ message.content }}</div>
  {% endfor %}
</div>

<form id="chat-form">
  {% csrf_token %}
  <input name="content" id="chat-input" autocomplete="off" required>
  <button type="submit">Send</button>
</form>

<script>
document.getElementById("chat-form").addEventListener("submit", async (e) => {
  e.preventDefault();
  const input = document.getElementById("chat-input");
  const text = input.value;
  const messages = document.getElementById("messages");

  messages.insertAdjacentHTML("beforeend", `<div class="msg msg-user">${text}</div>`);
  const replyEl = document.createElement("div");
  replyEl.className = "msg msg-assistant";
  messages.appendChild(replyEl);
  input.value = "";

  const resp = await fetch(`/chat/{{ conversation.id }}/send/`, {
    method: "POST",
    headers: {
      "X-CSRFToken": document.querySelector("[name=csrfmiddlewaretoken]").value,
      "Content-Type": "application/x-www-form-urlencoded",
    },
    body: `content=${encodeURIComponent(text)}`,
  });

  const reader = resp.body.getReader();
  const decoder = new TextDecoder();
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    replyEl.textContent += decoder.decode(value, { stream: true });
    messages.scrollTop = messages.scrollHeight;
  }
});
</script>

This is intentionally framework-free. If you’d rather avoid hand-rolled fetch code, htmx’s SSE extension does the same job declaratively — but the pattern of “append user message, open a stream, append tokens as they land” stays the same either way.

Step 6: handle failures without breaking the conversation

Chat completions calls fail — rate limits, timeouts, a provider having a bad day. Wrap the stream in a try/except that degrades gracefully instead of leaving the user staring at a stuck spinner:

def event_stream():
    chunks = []
    try:
        for delta in stream_reply(conversation):
            chunks.append(delta)
            yield delta
    except Exception:
        fallback = "Sorry — something went wrong generating a reply. Please try again."
        yield fallback
        chunks = [fallback]
    finally:
        full_text = "".join(chunks)
        if full_text:
            Message.objects.create(conversation=conversation, role="assistant", content=full_text)

If you’re calling a single provider directly, that’s the extent of your error handling — a provider outage just means downtime for your chat feature until it recovers. Routing through a gateway like n4n.ai changes the failure mode: because the same /v1/chat/completions call fans out across 240+ models with automatic fallback, a single provider hiccup gets retried against another model in the pool before your except block ever fires, so end users see a slower response rather than an error message.

Step 7: keep an eye on cost

Pay-per-token pricing means chat features have a variable cost per conversation, which is easy to lose track of once the feature ships. Two habits keep it manageable:

None of this requires new infrastructure — it’s mostly a matter of setting sane defaults in services.py and revisiting LLM_MODEL once you have real usage data.

Wrapping up

You now have a working chat feature: a Conversation/Message schema, a service layer that streams completions, a Django view that persists history even on disconnect, and a minimal frontend that renders tokens as they arrive. From here, the natural next steps are per-user rate limiting, conversation titling (ask the model to summarize the first exchange), and system prompts scoped to your app’s domain. Because the integration is a single OpenAI-compatible client pointed at a base_url, none of those additions require touching your models or views — you’re just extending the same stream_reply function.