Topic

Multimodal & Voice Apps with AI Frameworks

13 posts on multimodal & voice apps with ai frameworks — part of framework tutorials on the n4n AI blog.

Voice-to-text pipelines with Deepgram and LlamaIndex

Build a production-ready voice-to-text pipeline using Deepgram for transcription and LlamaIndex for retrieval-augmented querying over audio content.

July 27, 20263 min read

Framework tutorialsTutorial

Vision-language chat with Claude 3.5 Sonnet in LangChain

Build a production-ready vision chat application using Claude 3.5 Sonnet and LangChain with streaming, multi-image support, and error handling.

July 27, 20262 min read

Framework tutorialsHow-to

Transcribe and summarize calls with Whisper and LangChain

Build a production-ready pipeline to transcribe audio calls with Whisper and summarize them using LangChain, with code you can run today.

July 27, 20263 min read

Framework tutorialsComparison

Pipecat vs LiveKit Agents for building voice AI apps

Technical comparison of Pipecat and LiveKit Agents for voice AI applications, covering architecture, latency, pricing, and when to choose each framework.

July 27, 20267 min read

Framework tutorialsTutorial

Parse multimodal documents with LlamaIndex and GPT-4o

Build a production-ready multimodal document parser using LlamaIndex and GPT-4o with code examples for PDFs, images, and mixed-content files.

July 27, 20263 min read

Framework tutorialsTutorial

Multimodal agents in LangGraph with GPT-4o vision

Build a production-ready multimodal agent with LangGraph and GPT-4o vision — handles images, maintains state, and routes dynamically.

July 27, 20263 min read

Framework tutorialsHow-to

Build a phone agent with Pipecat, Deepgram, and ElevenLabs

Build a production-ready phone agent using Pipecat, Deepgram STT, and ElevenLabs TTS with Twilio integration. Complete tutorial with runnable code and verification steps.

July 27, 20264 min read

Framework tutorialsGuide

Speech-to-speech apps with the OpenAI Realtime API

A practical openai realtime api speech-to-speech tutorial: build low-latency voice apps with WebSockets, handle audio, and avoid common pitfalls.

July 19, 20263 min read

Framework tutorialsHow-to

Real-time voice apps with LiveKit Agents and Whisper

Step-by-step guide to building a production-ready real-time voice app with LiveKit Agents and Whisper STT, including LLM wiring and TTS.

July 19, 20264 min read

Framework tutorialsTutorial

Multimodal RAG in LlamaIndex with GPT-4o and images

Build a working multimodal RAG pipeline in LlamaIndex with GPT-4o and images. Step-by-step code for indexing and querying visual documents.

July 19, 20263 min read

Framework tutorialsHow-to

Build a voice bot with Vapi and Twilio integration

Step-by-step vapi twilio voice bot integration tutorial: connect Vapi assistants to Twilio telephony and a custom LLM endpoint in production.

July 19, 20264 min read

Framework tutorialsHow-to

Build a voice assistant with Pipecat and GPT-4o Realtime

Step-by-step pipecat gpt-4o realtime voice assistant tutorial: build a low-latency voice agent with Pipecat and OpenAI Realtime, from install to verification.

July 19, 20264 min read

Framework tutorialsTutorial

Analyze images and PDFs with Gemini 2.0 in LangChain

Hands-on tutorial: build multimodal pipelines with Gemini 2.0 and LangChain to analyze images and PDFs locally, with runnable code and expected outputs.

July 19, 20263 min read