Building a Slack support bot with LangChain and RAG

A slack support bot langchain rag pipeline turns your existing documentation into answers inside the channel your users already live in. This tutorial builds a production-shaped bot that ingests Markdown docs, retrieves relevant chunks, and replies with citations—no custom model training required.

Prerequisites

Python 3.11 or newer
A Slack workspace where you can install apps
A Slack bot token (xoxb-...), signing secret, and Socket Mode app token (xapp-...)
Bot scopes: app_mentions:read, chat:write, im:history, channels:history
A folder of Markdown docs (e.g., ./docs)
An API key for an OpenAI-compatible LLM endpoint. We’ll point LangChain at n4n.ai’s single OpenAI-compatible endpoint to reach 240+ models with automatic fallback when a provider is degraded.
Install dependencies:

pip install langchain langchain-openai langchain-community faiss-cpu slack-bolt python-dotenv

Create a .env file:

SLACK_BOT_TOKEN=xoxb-...
SLACK_SIGNING_SECRET=...
SLACK_APP_TOKEN=xapp-...
LLM_API_KEY=sk-...
LLM_BASE_URL=https://api.n4n.ai/v1
EMBEDDING_MODEL=text-emding-3-small
CHAT_MODEL=gpt-4o-mini

Load and chunk your docs

LangChain’s DirectoryLoader reads files; MarkdownTextSplitter respects heading boundaries. Keep chunks near 1,000 characters so retrieval stays precise and context windows are not wasted.

from langchain_community.document_loaders import DirectoryLoader
from langchain_text_splitters import MarkdownTextSplitter

loader = DirectoryLoader("./docs", glob="**/*.md")
docs = loader.load()

splitter = MarkdownTextSplitter(chunk_size=1000, chunk_overlap=200)
chunks = splitter.split_documents(docs)
print(f"Loaded {len(docs)} files, split into {len(chunks)} chunks")
print("Sample chunk:", chunks[0].page_content[:120])

Expected output:

Loaded 12 files, split into 147 chunks
Sample chunk: # API Keys

To create an API key, navigate to Dashboard > Settings > Tokens and click "Generate".

Embed and index with FAISS

Use OpenAIEmbeddings pointed at the gateway. Because the endpoint is OpenAI-compatible, no LangChain fork is needed—just set base_url. FAISS builds an in-memory vector store; swap to a persistent store for scale.

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
import os

embeddings = OpenAIEmbeddings(
    model=os.environ["EMBEDDING_MODEL"],
    api_key=os.environ["LLM_API_KEY"],
    base_url=os.environ["LLM_BASE_URL"],
)

vectorstore = FAISS.from_documents(chunks, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})
print("Index ready, dim:", vectorstore.index.d)

Expected output:

Index ready, dim: 1536

Build the RAG chain

We use create_retrieval_chain with a strict prompt that forces answers from retrieved context only. The chat model also goes through the gateway, so a provider outage triggers automatic fallback without code changes. Defining the prompt locally avoids a runtime fetch from a hub.

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain

llm = ChatOpenAI(
    model=os.environ["CHAT_MODEL"],
    api_key=os.environ["LLM_API_KEY"],
    base_url=os.environ["LLM_BASE_URL"],
    temperature=0,
)

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a support agent. Answer using only the context. "
               "If unsure, say you don't know. Cite source filenames."),
    ("human", "Context:\n{context}\n\nQuestion: {input}"),
])

combine_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, combine_chain)

response = rag_chain.invoke({"input": "How do I reset my API key?"})
print(response["answer"])
print("Sources:", [d.metadata.get("source") for d in response["context"]])

Expected output (truncated):

You can reset your API key from the dashboard under Settings > Tokens.
Sources: ['./docs/account.md', './docs/security.md']

Wire up Slack with Bolt

Slack’s Bolt framework handles verification and retries. We listen for message events, ignore bot messages, run the RAG chain, and post a threaded reply. Keep tokens in env; never hardcode.

from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
from dotenv import load_dotenv

load_dotenv()

app = App(
    token=os.environ["SLACK_BOT_TOKEN"],
    signing_secret=os.environ["SLACK_SIGNING_SECRET"],
)

@app.event("message")
def handle_message(event, say):
    if event.get("bot_id"):
        return
    user_q = event["text"]
    resp = rag_chain.invoke({"input": user_q})
    answer = resp["answer"]
    sources = {d.metadata.get("source") for d in resp["context"]}
    say(
        text=f"{answer}\n\n_Sources: {', '.join(sources)}_",
        thread_ts=event.get("ts"),
    )

if __name__ == "__main__":
    handler = SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"])
    handler.start()

Run and test

Start the script:

python bot.py

In Slack, send a message to the bot’s DM or a channel where it’s added:

@supportbot how do I rotate my API key?

Bot replies in thread:

You can rotate your API key from Settings > Tokens > Rotate. The old key expires in 24h.

_Sources: ./docs/account.md, ./docs/security.md_

Production notes

Persist FAISS to disk with vectorstore.save_local("faiss_index") and reload with FAISS.load_local so you don’t re-embed on every restart.
Add rate limiting on the Slack event handler to avoid LLM throttling during spikes.
Use n4n.ai’s per-token metering to attribute costs per Slack workspace if you run multi-tenant support.
Honor provider cache-control hints by passing extra_headers through LangChain’s model_kwargs when your gateway forwards them.
For non-Socket Mode deployments, use slack_bolt.adapter.flask.SlackRequestHandler behind a reverse proxy with proper SSL.

That is a complete slack support bot langchain rag deployment. Extend the retriever with metadata filters (e.g., by product area) to narrow answers for large doc sets, and add a feedback emoji reaction to collect failing queries for eval.