Building a Slack-based AI agent for incident commanders

A Slack AI agent incident commander reduces mean-time-to-acknowledge by putting triage logic where responders already live. This tutorial builds a Python service that listens for Slack events, calls an LLM to classify and summarize incidents, and posts structured updates back to the channel.

Prerequisites

• Python 3.11 or newer
• A Slack workspace where you can install apps
• A Slack app with a bot token (xoxb-...) and Event Subscriptions enabled for message.channels
• ngrok to expose your local server to Slack
• An OpenAI-compatible API key; we point the client at n4n.ai’s gateway for automatic fallback across providers

Install dependencies:

pip install slack-bolt openai fastapi uvicorn

1. Scaffold the Slack event listener

Slack Bolt handles verification and retries. Wrap it in FastAPI to receive events over HTTP.

from slack_bolt.app import App
from slack_bolt.adapter.fastapi import SlackRequestHandler
from fastapi import FastAPI, Request

app = App(token="xoxb-your-token", signing_secret="your-signing-secret")
api = FastAPI()
handler = SlackRequestHandler(app)

@api.post("/slack/events")
async def slack_events(req: Request):
    return handler.handle(req)

Run the server:

uvicorn main:api --port 3000

Expose it:

ngrok http 3000

Paste the ngrok URL + /slack/events into the Slack app’s Event Subscriptions field. Slack sends a url_verification challenge; Bolt answers automatically. Expected checkpoint: your ngrok log shows a POST with 200 OK and no errors.

2. Detect incident messages

We ignore bots and only act on messages containing operational keywords. This prevents the Slack AI agent incident commander from reacting to casual chatter.

INCIDENT_KEYWORDS = {"sev1", "sev2", "outage", "paging", "alert"}

@app.event("message")
def handle_message(event, say):
    if event.get("subtype") == "bot_message":
        return
    text = event.get("text", "").lower()
    if not any(k in text for k in INCIDENT_KEYWORDS):
        return
    process_incident(event, say)

Note: Slack delivers the same event multiple times on retry. In production, dedupe on event_id with Redis or a local lock. For this tutorial, keyword gating is enough.

3. Call the LLM for triage

Configure the OpenAI client against the OpenAI-compatible endpoint. n4n.ai forwards provider cache-control hints and meters per token, so you see exact usage per incident.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.n4n.ai/v1",
    api_key="sk-your-key",
)

def summarize_incident(raw_text: str) -> str:
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are an incident commander. Return JSON with: severity, service, timeline, first_action."},
            {"role": "user", "content": raw_text}
        ],
        response_format={"type": "json_object"}
    )
    return resp.choices[0].message.content

The model returns strict JSON. Example output for input "alert: checkout-api latency spike, started 14:02":

{
  "severity": "SEV2",
  "service": "checkout-api",
  "timeline": "14:02 latency alert fired",
  "first_action": "Check recent deploy and rollback if error rate climbs"
}

Because the gateway addresses 240+ models, you can swap model to claude-3-haiku or others without code changes. The fallback kicks in if the primary provider is degraded.

4. Post structured commander updates

Thread the response under the original message so the channel stays readable.

def process_incident(event, say):
    channel = event["channel"]
    ts = event["ts"]
    text = event["text"]
    analysis = summarize_incident(text)
    say(
        text=f":rotating_light: *Incident Commander Update*\n```{analysis}```",
        thread_ts=ts,
    )

Checkpoint: post in Slack:

alert: checkout-api latency spike, started 14:02

The bot replies in-thread:

:rotating_light: Incident Commander Update
{
  "severity": "SEV2",
  "service": "checkout-api",
  "timeline": "14:02 latency alert fired",
  "first_action": "Check recent deploy and rollback if error rate climbs"
}

That is the core Slack AI agent incident commander loop.

5. Add a resolve command

Give commanders a way to close the loop. Register a slash command in Slack, then handle it:

@app.command("/ic-resolve")
def resolve(ack, command, say):
    ack()
    say(f":white_check_mark: Incident resolved by {command['user_name']}")

Slack requires /ic-resolve to be configured with the same ngrok URL under Slash Commands. After typing /ic-resolve in the channel, the bot posts the confirmation.

Operational hardening

The toy above works locally. For production, add:

• Idempotency: store event_id in a TTL cache. Slack retries for up to 3 seconds.
• Secrets: load tokens from environment variables, not source.
• Timeouts: set client.chat.completions.create(timeout=10) so a slow LLM doesn’t block the event ack.
• Structured logging: log the channel, ts, and model used for each incident.

If you route through n4n.ai, per-token metering shows up in your usage dashboard, making it easy to attribute cost to specific Slack channels.

Extending the agent

The pattern generalizes. You can:

• Feed the last 50 channel messages into the prompt to build a running timeline.
• Use Slack’s blocks API to render severity as a colored attachment.
• Trigger PagerDuty or Opsgenie from the extracted severity field.

The Slack AI agent incident commander is now a real component: it listens, classifies, and reports without leaving the responder’s window.