Most LLM chat features die in production because the storage layer wasn’t thought through. If you’re building on Django, the right django models llm conversation history design determines whether you can debug sessions, bill accurately, and replay context without pulling your hair out. This guide walks through a concrete schema and the write paths that keep it consistent under real traffic.
1. Split conversations from messages
The fastest way to paint yourself into a corner is a single Conversation table with a messages JSON column. You lose the ability to index by role, filter by date, or join usage data. Use two tables: one row per session, one row per turn.
from django.db import models
import uuid
class Conversation(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
user = models.ForeignKey('auth.User', on_delete=models.CASCADE, related_name='conversations')
title = models.CharField(max_length=255, blank=True)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Message(models.Model):
ROLE_CHOICES = [
('system', 'system'),
('user', 'user'),
('assistant', 'assistant'),
('tool', 'tool'),
]
id = models.BigAutoField(primary_key=True)
conversation = models.ForeignKey(Conversation, on_delete=models.CASCADE, related_name='messages')
role = models.CharField(max_length=16, choices=ROLE_CHOICES)
content = models.TextField()
created_at = models.DateTimeField(auto_now_add=True)
model = models.CharField(max_length=128, blank=True)
prompt_tokens = models.IntegerField(default=0)
completion_tokens = models.IntegerField(default=0)
provider = models.CharField(max_length=64, blank=True)Keep content as TextField. Postgres handles large strings fine, and you avoid JSON serialization overhead for plain text. Add a composite index on (conversation_id, id) if you routinely paginate older messages—Django creates the foreign-key index already, but explicit ordering matters for window queries.
2. Record token usage and routing metadata
Token counts are not optional if you ever need to bill or rate-limit. Store them per message, not per conversation, because a session can span multiple models and providers. If you route through a gateway like n4n.ai, the response includes provider and token counts even when it fails over to a backup model; store those fields so metering matches reality.
A common mistake is trusting the model name you sent. The server may substitute a fallback. Persist the model and provider returned in the response body.
# after a non-streaming call
msg = Message(
conversation=conv,
role='assistant',
content=resp.choices[0].message.content,
model=resp.model,
prompt_tokens=resp.usage.prompt_tokens,
completion_tokens=resp.usage.completion_tokens,
provider=resp.headers.get('x-provider', 'unknown'),
)
msg.save()3. Write atomically around the LLM call
The naive flow—save user message, call API, save assistant message—breaks when the API throws after you’ve already committed the user turn. That leaves orphaned contexts that look incomplete. Wrap the assistant write in the same transaction as any state change, but not the user write (you want that regardless of LLM success).
from django.db import transaction
import openai
def send_message(conv, user_text):
Message.objects.create(conversation=conv, role='user', content=user_text)
messages = [
{'role': m.role, 'content': m.content}
for m in conv.messages.order_by('id')
]
try:
resp = openai.ChatCompletion.create(
model='gpt-4o-mini',
messages=messages,
)
except openai.APIError:
# user message already persisted; surface error to client
raise
with transaction.atomic():
Message.objects.create(
conversation=conv,
role='assistant',
content=resp.choices[0].message.content,
model=resp.model,
prompt_tokens=resp.usage.prompt_tokens,
completion_tokens=resp.usage.completion_tokens,
)Streaming changes the equation
With stream=True, you don’t have final content or usage until the last chunk. Writing to the database on every delta is a write-amplification disaster. Accumulate in memory, then do one update.
def send_message_stream(conv, user_text):
Message.objects.create(conversation=conv, role='user', content=user_text)
assistant = Message(conversation=conv, role='assistant', content='')
assistant.save()
buf = []
for chunk in openai.ChatCompletion.create(model='gpt-4o-mini', messages=[...], stream=True):
delta = chunk.choices[0].delta.get('content', '')
if delta:
buf.append(delta)
assistant.content = ''.join(buf)
assistant.save(update_fields=['content'])If you need live UI updates, push deltas over websockets; keep the DB write final.
4. Handle tool calls and structured output
Modern LLM APIs return tool_calls alongside or instead of content. A TextField can’t hold that. Add a nullable JSONField.
class Message(models.Model):
# ... previous fields ...
tool_calls = models.JSONField(null=True, blank=True)
tool_call_id = models.CharField(max_length=64, blank=True)When the assistant requests a tool, store the call JSON. When your code executes it, store the result as a tool role message referencing tool_call_id. This keeps the transcript replayable for models that require strict ordering.
5. Assemble context without killing the database
The simplest context build is conversation.messages.all(). That fetches the entire history on every request—fine at session start, fatal at message 500. Bound it.
def build_context(conv, max_tokens=8000, hard_limit=40):
msgs = conv.messages.order_by('-id')[:hard_limit].reverse()
context = []
total = 0
for m in msgs:
# rough estimate: 4 chars ~= 1 token
est = len(m.content) // 4 + (m.tool_calls and 200 or 0)
if total + est > max_tokens:
break
total += est
context.append({'role': m.role, 'content': m.content})
return contextUse select_related if you display conversation metadata in the same view. Avoid prefetch_related for messages when you’re slicing—Postgres will still materialize the join.
6. Pitfalls and tradeoffs
JSON blobs for history. Teams often start with a single JSONField to move fast. You can’t query “show me all conversations where the assistant mentioned X” without brute-force scans. Migrating later means backfilling thousands of rows through a Python script. The two-table approach costs one extra migration and saves weeks.
Token drift. Provider token counts are approximate across versions. If you store them, treat them as authoritative for that provider only. Don’t sum prompt_tokens from OpenAI and a local model and call it billing.
Streaming race conditions. If you write a draft assistant row before streaming completes, a second concurrent request from the same user can read that empty row as context. Use a status field (pending, complete) and filter it out, or lock the conversation with select_for_update() when building context.
Cascade deletes. on_delete=CASCADE is correct, but ensure your S3/object storage for any attached files uses the same signal, or you’ll leak blobs.
Provider fallback masking. When a gateway silently swaps models, your model field must reflect what actually generated the text. Otherwise debugging “why did this response suck” becomes impossible.
Indexing role. If you frequently filter role='system', add a partial index:
class Meta:
indexes = [
models.Index(fields=['conversation', 'id'], name='conv_msg_order'),
models.Index(fields=['role'], name='msg_role', condition=models.Q(role='system')),
]7. Migrating an existing single-table design
If you shipped the JSON column already, write a one-time management command. Stream the old rows, parse the array, and bulk_create Message objects in batches of 500 inside transactions. Keep the old column for a release cycle, then drop it. Don’t attempt this in a single migration on a table with millions of rows—you’ll lock it.
from django.core.management.base import BaseCommand
from myapp.models import Conversation, Message
class Command(BaseCommand):
def handle(self, *args, **options):
for conv in Conversation.objects.exclude(history__isnull=True).iterator(chunk_size=200):
msgs = [Message(conversation=conv, role=m['role'], content=m['content']) for m in conv.history]
Message.objects.bulk_create(msgs)The django models llm conversation history pattern above is boring on purpose. Boring storage is what lets you ship tool-calling, streaming, and multi-provider routing without rewriting your data layer every quarter.