This tutorial walks you through building a complete retrieval-augmented generation (RAG) system using LangChain.js, Chroma, and Node.js. You’ll go from an empty directory to a working pipeline that loads documents, chunks them, stores embeddings in Chroma, and answers questions with cited sources. The code is written for TypeScript and runs on Node 18+.
Prerequisites
Before starting, make sure you have:
- Node.js 18 or later —
node --versionshould print v18.x or higher - npm or pnpm — package manager of choice
- An OpenAI API key — for embeddings and chat completion (or a compatible provider)
- Chroma running locally —
docker run -p 8000:8000 chromadb/chromagets you a local instance in seconds
You’ll also need a basic understanding of async/await in TypeScript and familiarity with LangChain’s core abstractions: documents, text splitters, embeddings, vector stores, and chains.
Project setup
Create a new directory and initialize a TypeScript project:
mkdir langchain-rag-chroma && cd langchain-rag-chroma
npm init -y
npm install langchain @langchain/community @langchain/openai chromadb
npm install -D typescript tsx @types/nodeConfigure TypeScript with a minimal tsconfig.json:
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"lib": ["ES2022"],
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"outDir": "./dist",
"rootDir": "./src"
},
"include": ["src/**/*"]
}Add a src directory and create an environment file for your API key:
mkdir src
echo "OPENAI_API_KEY=sk-your-key-here" > .envInstall dotenv to load it:
npm install dotenvLoading and splitting documents
Start with a simple document loader. For this tutorial we’ll use a local text file, but the same pattern works with PDFs, HTML, Notion, or any loader in @langchain/community/document_loaders.
Create src/load-documents.ts:
import "dotenv/config";
import { TextLoader } from "langchain/document_loaders/fs/text";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { Document } from "langchain/document";
async function loadAndSplit(): Promise<Document[]> {
const loader = new TextLoader("data/sample.txt");
const docs = await loader.load();
const splitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
separators: ["\n\n", "\n", ". ", " ", ""],
});
const chunks = await splitter.splitDocuments(docs);
console.log(`Loaded ${docs.length} document(s), split into ${chunks.length} chunks`);
return chunks;
}
loadAndSplit().catch(console.error);Create a sample document at data/sample.txt:
LangChain is a framework for developing applications powered by language models.
It provides abstractions for working with LLMs, prompts, memory, and agents.
Chroma is an open-source embedding database designed for LLM applications.
It stores vectors with metadata and supports similarity search with filtering.
Retrieval-augmented generation (RAG) combines information retrieval with text generation.
The retriever finds relevant documents, then the generator produces an answer grounded in those documents.
Embeddings convert text into high-dimensional vectors that capture semantic meaning.
Similar texts have similar vectors, enabling semantic search beyond keyword matching.Run the loader to verify it works:
npx tsx src/load-documents.tsExpected output:
Loaded 1 document(s), split into 4 chunksCreating the vector store
Now wire up Chroma. LangChain’s @langchain/community/vectorstores/chroma package handles the client connection and provides a Chroma class that implements the VectorStore interface.
Create src/create-vectorstore.ts:
import "dotenv/config";
import { OpenAIEmbeddings } from "@langchain/openai";
import { Chroma } from "@langchain/community/vectorstores/chroma";
import { loadAndSplit } from "./load-documents";
async function createVectorStore() {
const chunks = await loadAndSplit();
const embeddings = new OpenAIEmbeddings({
modelName: "text-embedding-3-small",
});
const vectorStore = await Chroma.fromDocuments(chunks, embeddings, {
collectionName: "langchain-rag-tutorial",
url: "http://localhost:8000", // default Chroma port
collectionMetadata: {
"hnsw:space": "cosine",
},
});
console.log("Vector store created and documents indexed");
return vectorStore;
}
createVectorStore().catch(console.error);Run it:
npx tsx src/create-vectorstore.tsExpected output:
Loaded 1 document(s), split into 4 chunks
Vector store created and documents indexedAt this point, Chroma has four vectors stored in the langchain-rag-tutorial collection. You can verify with the Chroma REST API:
curl http://localhost:8000/api/v1/collections/langchain-rag-tutorial/getBuilding the retrieval chain
LangChain’s createRetrievalChain and createStuffDocumentsChain (from langchain/chains/combine_documents) compose the retriever and the generation step. We’ll use a prompt that instructs the model to cite sources.
Create src/query.ts:
import "dotenv/config";
import { OpenAIEmbeddings } from "@langchain/openai";
import { ChatOpenAI } from "@langchain/openai";
import { Chroma } from "@langchain/community/vectorstores/chroma";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { ChatPromptTemplate } from "@langchain/core/prompts";
const SYSTEM_PROMPT = `You are an assistant for question-answering tasks.
Use the following pieces of retrieved context to answer the question.
If you don't know the answer, say you don't know.
Cite your sources using the document metadata.
Context: {context}`;
async function runQuery(question: string) {
const embeddings = new OpenAIEmbeddings({
modelName: "text-embedding-3-small",
});
const vectorStore = await Chroma.fromExistingCollection(embeddings, {
collectionName: "langchain-rag-tutorial",
url: "http://localhost:8000",
});
const retriever = vectorStore.asRetriever({
k: 4,
});
const llm = new ChatOpenAI({
modelName: "gpt-4o-mini",
temperature: 0,
});
const prompt = ChatPromptTemplate.fromMessages([
["system", SYSTEM_PROMPT],
["human", "{input}"],
]);
const combineDocsChain = await createStuffDocumentsChain({
llm,
prompt,
});
const retrievalChain = await createRetrievalChain({
combineDocsChain,
retriever,
});
const result = await retrievalChain.invoke({
input: question,
});
return result;
}
const question = process.argv[2] || "What is RAG and how does it work?";
const result = await runQuery(question);
console.log("\n--- Answer ---");
console.log(result.answer);
console.log("\n--- Source Documents ---");
result.context.forEach((doc, i) => {
console.log(`[${i + 1}] ${doc.pageContent.slice(0, 120)}...`);
});Run a query:
npx tsx src/query.ts "What is Chroma and what does it store?"Expected output (abridged):
--- Answer ---
Chroma is an open-source embedding database designed for LLM applications. It stores vectors with metadata and supports similarity search with filtering. [1]
--- Source Documents ---
[1] Chroma is an open-source embedding database designed for LLM applications. It stores vectors with metadata and supports similarity search with filtering.The answer cites the source document by index. The context array contains the full retrieved chunks — useful for debugging or displaying citations in a UI.
Streaming responses
For a better user experience, stream tokens as they arrive. LangChain’s stream method on the retrieval chain yields partial results.
Update src/query.ts to add a streaming variant:
async function runQueryStreaming(question: string) {
const embeddings = new OpenAIEmbeddings({
modelName: "text-embedding-3-small",
});
const vectorStore = await Chroma.fromExistingCollection(embeddings, {
collectionName: "langchain-rag-tutorial",
url: "http://localhost:8000",
});
const retriever = vectorStore.asRetriever({ k: 4 });
const llm = new ChatOpenAI({
modelName: "gpt-4o-mini",
temperature: 0,
streaming: true,
});
const prompt = ChatPromptTemplate.fromMessages([
["system", SYSTEM_PROMPT],
["human", "{input}"],
]);
const combineDocsChain = await createStuffDocumentsChain({ llm, prompt });
const retrievalChain = await createRetrievalChain({ combineDocsChain, retriever });
console.log("\n--- Streaming Answer ---\n");
const stream = await retrievalChain.stream({ input: question });
for await (const chunk of stream) {
if (chunk.answer) {
process.stdout.write(chunk.answer);
}
}
console.log("\n");
}
const mode = process.argv[2] === "--stream" ? "stream" : "invoke";
const question = process.argv[mode === "stream" ? 3 : 2] || "What is RAG?";
if (mode === "stream") {
await runQueryStreaming(question);
} else {
const result = await runQuery(question);
console.log("\n--- Answer ---");
console.log(result.answer);
console.log("\n--- Source Documents ---");
result.context.forEach((doc, i) => {
console.log(`[${i + 1}] ${doc.pageContent.slice(0, 120)}...`);
});
}Run with streaming:
npx tsx src/query.ts --stream "Explain embeddings in one sentence"Output appears token by token:
--- Streaming Answer ---
Embeddings convert text into high-dimensional vectors that capture semantic meaning, enabling similarity search beyond keyword matching.Adding metadata filtering
Chroma supports filtering by metadata at query time. This is essential when you have multiple tenants, document types, or time ranges. Modify the retriever to include a filter option:
const retriever = vectorStore.asRetriever({
k: 4,
filter: {
source: "data/sample.txt", // only search documents from this source
},
});Filters use Chroma’s where clause syntax. For complex filters, pass a Where object:
filter: {
$and: [
{ source: { $eq: "data/sample.txt" } },
{ chunkIndex: { $gte: 0 } },
],
},Metadata is attached automatically by the document loader. For PDFs, you’ll get pageNumber, source, and totalPages. For custom loaders, add metadata manually before splitting:
const docs = await loader.load();
docs.forEach((doc, i) => {
doc.metadata = {
...doc.metadata,
docId: `doc-${i}`,
ingestedAt: new Date().toISOString(),
};
});Persisting and reusing the vector store
Chroma.fromDocuments creates a new collection each run. In production, separate ingestion from querying. The ingestion script (your create-vectorstore.ts) runs once or on a schedule. The query script uses fromExistingCollection as shown in query.ts.
To delete and recreate a collection during development:
import { ChromaClient } from "chromadb";
const client = new ChromaClient({ path: "http://localhost:8000" });
await client.deleteCollection({ name: "langchain-rag-tutorial" });Add this to a src/reset.ts for quick iteration.
Production considerations
Embedding model choice
text-embedding-3-small (1536 dimensions) is cost-effective and fast. For higher quality, use text-embedding-3-large (3072 dimensions) — but expect 5x the embedding cost and larger index size. Test both on your data.
Chunk size and overlap
The 1000/200 split works for general prose. For code, use smaller chunks (500/100) with a code-aware splitter. For legal or technical docs, larger chunks (1500/300) preserve context. Measure retrieval quality with a small eval set before committing.
Retriever tuning
k: 4 is a starting point. Increase for broad questions, decrease for precise lookups. Add a scoreThreshold to filter low-similarity results:
const retriever = vectorStore.asRetriever({
k: 6,
searchType: "similarity_score_threshold",
scoreThreshold: 0.7,
});Handling rate limits and provider failures
If you route through a gateway that supports automatic fallback (like n4n.ai), you can swap the ChatOpenAI and OpenAIEmbeddings constructors to point at a single endpoint that handles provider failover transparently. The rest of your chain stays unchanged.
Monitoring
Log every query with: question, retrieved chunk IDs, similarity scores, latency, token usage, and the final answer. This lets you measure retrieval precision, catch hallucinations, and track cost per query.
Next steps
You now have a working RAG pipeline. From here, consider:
- Hybrid search — combine vector similarity with BM25 keyword search using
Chroma.fromDocumentswith a custom retriever - Query rewriting — use an LLM to expand or decompose the user question before retrieval
- Reranking — pass retrieved chunks through a cross-encoder (e.g.,
cohere/rerank-v3.5) before stuffing into the prompt - Evaluation — build a small golden set of Q&A pairs and measure answer correctness with an LLM judge
- Multi-tenancy — namespace collections by tenant ID or use Chroma’s metadata filtering with strict
whereclauses
The complete source for this tutorial is structured as:
src/
load-documents.ts # document loading + splitting
create-vectorstore.ts # ingestion pipeline
query.ts # retrieval + generation (invoke + stream)
reset.ts # development helper
data/
sample.txt # example corpusRun the full flow:
npx tsx src/create-vectorstore.ts
npx tsx src/query.ts "How does Chroma support filtering?"
npx tsx src/query.ts --stream "Summarize the relationship between LangChain and Chroma"You have a foundation that scales from prototype to production. The abstractions — loaders, splitters, embeddings, vector stores, retrievers, chains — are composable. Swap any piece without rewriting the rest.