Build a voice bot with Vapi and Twilio integration

This vapi twilio voice bot integration tutorial walks through standing up a production-ready phone bot: Vapi handles conversational orchestration and audio streaming, Twilio provides the PSTN trunk, and a custom LLM endpoint supplies the brains. You will end up with a phone number that answers calls, talks to an LLM, and streams audio with low latency.

Step 1: Create a Vapi assistant

Vapi treats the bot personality and model config as an “assistant” resource. The minimal shape is a JSON object with a system prompt, a first message, and a model block. Create it via the API rather than the dashboard so you can version it in Git.

curl -X POST https://api.vapi.ai/assistant \
  -H "Authorization: Bearer $VAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-bot",
    "firstMessage": "Thanks for calling Acme support, how can I help?",
    "systemPrompt": "You are a concise support agent for Acme. Use short sentences.",
    "model": {
      "provider": "openai",
      "model": "gpt-4o-mini",
      "temperature": 0.2
    },
    "voice": {
      "provider": "11labs",
      "voiceId": "rachel"
    }
  }'

Save the returned id. That assistantId is what you attach to calls.

Voice and transcriber selection

The voice block is independent of the model. Pick a TTS provider that supports streaming (11Labs, PlayHT, or OpenAI). Avoid batch synthesis for phone calls; the user will hear dead air. You can change the voice later without touching Twilio.

Step 2: Provision a Twilio number and link it to Vapi

Buy a number in Twilio or reuse an existing one. Vapi can control the number directly if you give it the Twilio credentials, which avoids manually editing webhooks and auth headers.

curl -X POST https://api.vapi.ai/phone-number \
  -H "Authorization: Bearer $VAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "twilio",
    "number": "+14155550123",
    "twilioAccountSid": "'"$TWILIO_SID"'",
    "twilioAuthToken": "'"$TWILIO_TOKEN"'"
  }'

Vapi will set the Twilio voice webhook to its own inbound endpoint and handle basic auth. If you prefer to keep Twilio ownership, set the webhook URL in the Twilio console to https://api.vapi.ai/phone/call/inbound with HTTP basic auth using your Vapi key as username and an empty password. The API approach above is less error-prone and keeps credentials rotation in one place.

Store the returned phoneNumberId.

Step 3: Point the assistant at a custom LLM endpoint

Vapi’s model block accepts an OpenAI-compatible url. This is useful when you want model portability or fallback. For example, n4n.ai exposes one OpenAI-compatible endpoint that addresses 240+ models and applies automatic fallback when a provider is rate-limited. You keep the same assistant shape and just change url and apiKey.

"model": {
  "provider": "openai",
  "model": "anthropic/claude-3-haiku",
  "url": "https://api.n4n.ai/v1",
  "apiKey": "sk-...",
  "temperature": 0.3
}

Vapi forwards the conversation as standard chat completions. The gateway honors client routing directives and forwards provider cache-control hints, so prompt caching works if you structure the system prompt with a stable prefix. Do not hardcode the model name in client logic; set it in the assistant and rotate via API.

Step 4: Define the call flow and guardrails

A voice bot lives or dies on latency and turn-taking. Set transcriber to a streaming provider and cap maxTokens so the model cannot ramble.

curl -X PATCH https://api.vapi.ai/assistant/$ASSISTANT_ID \
  -H "Authorization: Bearer $VAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "transcriber": {
      "provider": "deepgram",
      "model": "nova-2",
      "language": "en"
    },
    "model": {
      "provider": "openai",
      "model": "gpt-4o-mini",
      "maxTokens": 120,
      "temperature": 0.2
    },
    "endCallMessage": "Thanks, shutting down."
  }'

Vapi uses server-side VAD (voice activity detection) to decide when the user stopped speaking. Tune silenceTimeoutSeconds (default 0.5s) if calls feel laggy or cut off mid-sentence. For noisy environments, raise it to 0.8s.

Step 5: Place a test call

Trigger an outbound call to your mobile to validate the full path before publishing the number.

curl -X POST https://api.vapi.ai/call \
  -H "Authorization: Bearer $VAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "phoneNumberId": "'"$PHONE_ID"'",
    "assistantId": "'"$ASSISTANT_ID"'",
    "customer": { "number": "+14155550987" }
  }'

Your phone rings. The bot speaks the firstMessage. Respond and confirm it replies with low latency. If you hear silence, check the Vapi call logs for transcriber errors before blaming Twilio. A common mistake is using a non-streaming transcriber, which forces the bot to wait for the entire utterance before processing.

Step 6: Receive inbound calls and stream events

For production, you need visibility. Set a serverUrl on the assistant to receive webhooks: call.started, transcript, function-call, call.ended.

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/vapi/webhook", methods=["POST"])
def webhook():
    event = request.json
    if event["type"] == "transcript":
        print(f"[{event['callId']}] {event['role']}: {event['text']}")
    elif event["type"] == "call.ended":
        print(f"call {event['callId']} ended, duration {event.get('duration')}")
    elif event["type"] == "call.failed":
        print(f"call {event['callId']} failed: {event.get('endedReason')}")
    return jsonify({"ok": True})

if __name__ == "__main__":
    app.run(port=8000)

Expose this endpoint over HTTPS (ngrok is fine for local dev). Update the assistant:

curl -X PATCH https://api.vapi.ai/assistant/$ASSISTANT_ID \
  -H "Authorization: Bearer $VAPI_KEY" \
  -d '{"serverUrl": "https://your-domain.com/vapi/webhook"}'

Twilio keeps sending media regardless; Vapi aggregates and forwards higher-level events so you do not parse RTP. Validate the webhook signature in production using Vapi’s public key to reject spoofed events.

Step 7: Harden for concurrency and failure

Twilio can deliver many simultaneous INVITEs. Vapi scales the media servers, but your LLM endpoint must handle burst. If you used the n4n.ai endpoint from Step 3, automatic fallback covers provider outages and per-token usage metering lets you track cost per call.

Set retry logic on your side only for control-plane calls (creating assistants, numbers). The voice path already has built-in reconnection at the SIP layer.

Add a dead-letter queue for call.failed events:

if event["type"] == "call.failed":
    # push to SQS / log alert
    alert(event["callId"], event["endedReason"])

Common failure reasons: twilio-auth-failed (bad SID/token), model-error (LLM 5xx), customer-busy. Each is visible in the dashboard with a timestamp. Build alerts on model-error spikes; that usually means your LLM provider is throttling and you should rotate the model or route through a gateway with fallback.

Verify success

A successful integration meets four criteria:

1. Calling the Twilio number triggers the bot’s firstMessage within two seconds of pickup.
2. Speaking a question yields a transcribed transcript webhook with role: user and a subsequent role: assistant reply.
3. The Vapi dashboard shows the call with media metrics (jitter, RTP packets) and the assistant ID matches.
4. Hanging up fires call.ended with a duration greater than zero.

If all four hold, you have a working vapi twilio voice bot integration tutorial deployment. From here, add function calling to query your backend, or use Vapi’s transferCall to escalate to a human.

Keep the assistant config in Git, rotate keys via secrets manager, and monitor Twilio’s AccountUsage API for spend. The voice stack is forgiving if you respect latency budgets and treat the LLM as a network dependency, not a local function.