How AI customer support agents handle ticket triage

AI customer support agents ticket triage is the first real test of any help-desk automation: route the incoming message to the right queue, assign priority, and extract structured context before a human touches it. Get this wrong and you bury your agents in misrouted noise; get it right and the rest of the pipeline stays clean.

Below is an end-to-end build of a triage service that runs in production. It assumes you have a help desk (Zendesk, Freshdesk, or your own DB) and an OpenAI-compatible LLM endpoint.

Step 1: Normalize the inbound ticket shape

Raw webhooks are messy. One vendor sends subject/body, another sends title/text. Define a single internal schema before any LLM call.

from pydantic import BaseModel, EmailStr
from typing import Optional

class InboundTicket(BaseModel):
    ticket_id: str
    subject: str
    body: str
    requester_email: EmailStr
    created_at: int  # epoch seconds
    raw_tags: list[str] = []

Expose a small HTTP receiver that maps external payloads into this model:

from fastapi import FastAPI, Request
app = FastAPI()

@app.post("/webhook/zendesk")
async def zendesk_webhook(req: Request):
    data = await req.json()
    ticket = InboundTicket(
        ticket_id=str(data["id"]),
        subject=data["subject"],
        body=data["description"],
        requester_email=data["requester"]["email"],
        created_at=int(data["created_at"]),
        raw_tags=[t["name"] for t in data.get("tags", [])]
    )
    # hand off to triage queue
    await triage_queue.put(ticket)
    return {"ok": True}

Verify: Send a canned JSON payload with curl to the endpoint. Assert the service logs a normalized InboundTicket and acknowledges with {"ok": true}.

Step 2: Call an LLM for structured triage decisions

The core of AI customer support agents ticket triage is a single structured completion. Use JSON mode to force a parseable response. The prompt must specify the allowed queues and priority levels.

import json
from openai import OpenAI

# Point at any OpenAI-compatible gateway; this same code works with n4n.ai,
# which adds automatic fallback when a provider is degraded.
client = OpenAI(base_url="https://api.n4n.ai/v1", api_key="sk-yourkey")

TRIAGE_SYSTEM = """You are a support triage classifier. Given a ticket, output JSON:
{
  "queue": "billing|tech|account|sales|spam",
  "priority": "p1|p2|p3",
  "language": "en|es|fr|de|other",
  "summary": "one-line summary",
  "entities": {"order_id": null, "plan": null}
}
Only use provided queues. If unsure, pick the closest and set priority p3."""

def classify(t: InboundTicket) -> dict:
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": TRIAGE_SYSTEM},
            {"role": "user", "content": f"Subject: {t.subject}\nBody: {t.body}"}
        ],
        temperature=0.0
    )
    return json.loads(resp.choices[0].message.content)

Keep temperature=0 for deterministic routing. The model returns a dict you can validate with Pydantic.

Verify: Run classify() on three sample tickets (billing, tech, spam). Print the JSON. Confirm queue is one of the enum values and priority is present.

Step 3: Merge model output with deterministic business rules

Never let the model alone decide SLA-critical routing. Apply hard rules after the call:

def apply_rules(t: InboundTicket, llm_out: dict) -> dict:
    # VIP customers always p1
    if t.requester_email.split("@")[1] in {"acme-corp.com", "keyclient.io"}:
        llm_out["priority"] = "p1"
        llm_out["queue"] = "account"
    # Explicit spam tag from vendor overrides model
    if "spam" in t.raw_tags:
        llm_out["queue"] = "spam"
        llm_out["priority"] = "p3"
    return llm_out

This layer is where AI customer support agents ticket triage becomes trustworthy: the LLM proposes, the rules dispose.

Verify: Feed a ticket from acme-corp.com with a billing question. Assert output priority is p1 and queue account regardless of model’s initial suggestion.

Step 4: Route and persist the triage result

Push the decision back to the help desk. Most APIs accept a PUT with tags and assignee group.

import httpx

def update_helpdesk(t: InboundTicket, triage: dict):
    payload = {
        "ticket": {
            "group": triage["queue"],
            "priority": triage["priority"],
            "tags": [f"triage:{triage['queue']}", f"lang:{triage['language']}"],
            "comment": {"body": f"Auto-triage: {triage['summary']}"}
        }
    }
    # example Zendesk update
    httpx.put(
        f"https://yourdomain.zendesk.com/api/v2/tickets/{t.ticket_id}.json",
        json=payload,
        auth=("api@domain.com", "token")
    )

If you use a gateway that honors client routing directives, you can pin specific models per queue (e.g., cheap model for spam, stronger for tech) by sending a header. n4n.ai forwards provider cache-control hints, so repeated similar tickets cost less.

Verify: After a test ticket, query the help desk API for that ticket. Confirm group matches triage queue and tags include triage:.

Step 5: Implement fallback and observability

LLM endpoints fail. Wrap the classify call in a timeout and fallback to keyword matching:

import re, asyncio

KEYWORD_MAP = {"refund": "billing", "bug": "tech", "upgrade": "sales"}

async def safe_classify(t: InboundTicket):
    try:
        return await asyncio.wait_for(classify_async(t), timeout=2.0)
    except Exception:
        queue = "tech"
        for kw, q in KEYWORD_MAP.items():
            if re.search(kw, t.body, re.I):
                queue = q
        return {"queue": queue, "priority": "p3", "language": "en",
                "summary": "fallback", "entities": {}}

# classify_async wraps the sync client in a thread

Log every triage decision with the model used and token count. Per-token metering (available on some gateways) lets you attribute cost per queue.

Verify: Kill the LLM endpoint temporarily. Send a ticket containing “refund”. Confirm fallback returns queue: billing and the event is logged as fallback.

Step 6: Verify success with offline eval and live shadowing

Before cutting humans out of the loop, run the pipeline against historical tickets where the true queue is known.

import csv
from collections import Counter

def eval_history(rows):
    correct = 0
    total = 0
    conf = Counter()
    for r in rows:
        t = InboundTicket(**r["input"])
        out = classify(t)
        out = apply_rules(t, out)
        total += 1
        if out["queue"] == r["expected_queue"]:
            correct += 1
        else:
            conf[(out["queue"], r["expected_queue"])] += 1
    print(f"accuracy {correct/total:.2%}")
    print("confusions", conf)

Aim for >90% queue accuracy on a held-out set. Then enable shadow mode: triage runs on live tickets but writes only a private note, not routing. Compare agent overrides for a week.

Verify: Produce the accuracy report from 500 past tickets. In shadow mode, check that <10% of agent actions contradict the auto-triage suggestion.

Closing notes

AI customer support agents ticket triage is not a single prompt; it is a pipeline with a strict schema, rule overrides, and fallback. The LLM does proposal, your code does enforcement. Ship the deterministic parts first, then widen model autonomy as eval numbers justify it.

That’s the build. The code blocks are minimal but runnable against any OpenAI-compatible endpoint; swap the base URL and your routing, caching, and metering come along for free.