Haystack document store tutorial: Weaviate setup guide

This haystack weaviate document store tutorial walks you through a production-ready setup: spinning up Weaviate, configuring the Haystack document store, indexing documents with embeddings, and running retrieval pipelines. We’ll cover the configuration details that matter in practice — batch sizing, hybrid search, and filtering — so you can move past the hello-world stage.

Prerequisites

You need Python 3.10+ and Docker. Install the Haystack Weaviate integration and an embedding model client:

pip install "haystack-ai[weaviate]" sentence-transformers

We’ll use sentence-transformers/all-MiniLM-L6-v2 locally for embeddings to keep the tutorial self-contained. In production you’d swap this for a hosted model (OpenAI, Cohere, or via n4n.ai’s OpenAI-compatible endpoint) without changing the pipeline structure.

Start Weaviate with Docker Compose. Create docker-compose.yml:

version: '3.4'
services:
  weaviate:
    image: cr.weaviate.io/semitechnologies/weaviate:1.25.3
    ports:
      - "8080:8080"
      - "50051:50051"
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'none'
      ENABLE_MODULES: ''
    volumes:
      - weaviate_data:/var/lib/weaviate

volumes:
  weaviate_data:

Bring it up:

docker compose up -d

Verify it’s healthy:

curl -s http://localhost:8080/v1/.well-known/ready
# Expected: {"status":"ready"}

Initialize the document store

Create setup_store.py:

from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,  # all-MiniLM-L6-v2 output dimension
    similarity="cosine",
    recreate_index=True,  # drop and recreate for clean tutorial runs
)

print(f"Document store ready. Index: {document_store.index}")
print(f"Collection exists: {document_store._collection_exists()}")

Run it:

python setup_store.py

Expected output:

Document store ready. Index: HaystackDocs
Collection exists: True

Key parameters explained:

• embedding_dim must match your embedder’s output dimension. all-MiniLM-L6-v2 produces 384-dimensional vectors.
• similarity="cosine" is the default and works well for normalized embeddings. Use "dot_product" only if you know your vectors aren’t normalized.
• recreate_index=True is convenient for tutorials; remove or gate behind a flag in production.

Create an indexing pipeline

We’ll build a pipeline that converts raw text files to documents, generates embeddings, and writes to Weaviate in batches. Create index_docs.py:

from pathlib import Path
from haystack import Pipeline, Document
from haystack.components.converters import TextFileToDocument
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.writers import DocumentWriter
from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore

# Sample data — replace with your own corpus
sample_docs = [
    Document(content="Haystack is an open-source LLM framework for building RAG pipelines."),
    Document(content="Weaviate is a vector database with hybrid search and GraphQL support."),
    Document(content="Embedding models convert text into dense vectors for semantic search."),
    Document(content="RAG combines retrieval with generation for grounded LLM answers."),
    Document(content="Batch indexing improves throughput by reducing network round-trips."),
]

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    similarity="cosine",
)

indexing_pipeline = Pipeline()
indexing_pipeline.add_component("embedder", SentenceTransformersDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"))
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store, policy="overwrite"))
indexing_pipeline.connect("embedder.documents", "writer.documents")

result = indexing_pipeline.run({"embedder": {"documents": sample_docs}})
print(f"Indexed {result['writer']['documents_written']} documents")

# Verify count
count = document_store.count_documents()
print(f"Documents in store: {count}")

Run it:

python index_docs.py

Expected output:

Indexed 5 documents
Documents in store: 5

Batch sizing in practice

The DocumentWriter accepts a batch_size parameter (default 1000). For large corpora, tune this based on your network latency and Weaviate’s import throughput. A safe starting point:

DocumentWriter(document_store=document_store, policy="overwrite", batch_size=256)

If you’re indexing millions of documents, consider the weaviate-client’s native batch import API directly — it bypasses Haystack’s overhead and supports parallel workers.

Build a retrieval pipeline

Now query the index. We’ll demonstrate three retrieval modes: dense vector search, keyword (BM25) search, and hybrid. Create retrieve.py:

from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.retrievers.weaviate import WeaviateEmbeddingRetriever, WeaviateBM25Retriever, WeaviateHybridRetriever
from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    similarity="cosine",
)

query = "What is RAG and how does it work?"

# Dense vector retrieval
dense_pipeline = Pipeline()
dense_pipeline.add_component("embedder", SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"))
dense_pipeline.add_component("retriever", WeaviateEmbeddingRetriever(document_store=document_store, top_k=3))
dense_pipeline.connect("embedder.embedding", "retriever.query_embedding")

dense_result = dense_pipeline.run({"embedder": {"text": query}})
print("=== Dense Retrieval ===")
for doc in dense_result["retriever"]["documents"]:
    print(f"  Score: {doc.score:.4f} | {doc.content[:80]}...")

# BM25 keyword retrieval
bm25_pipeline = Pipeline()
bm25_pipeline.add_component("retriever", WeaviateBM25Retriever(document_store=document_store, top_k=3))
bm25_result = bm25_pipeline.run({"retriever": {"query": query}})
print("\n=== BM25 Retrieval ===")
for doc in bm25_result["retriever"]["documents"]:
    print(f"  Score: {doc.score:.4f} | {doc.content[:80]}...")

# Hybrid retrieval (alpha=0.5 balances dense + sparse)
hybrid_pipeline = Pipeline()
hybrid_pipeline.add_component("embedder", SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"))
hybrid_pipeline.add_component("retriever", WeaviateHybridRetriever(document_store=document_store, top_k=3, alpha=0.5))
hybrid_pipeline.connect("embedder.embedding", "retriever.query_embedding")

hybrid_result = hybrid_pipeline.run({"embedder": {"text": query}, "retriever": {"query": query}})
print("\n=== Hybrid Retrieval (alpha=0.5) ===")
for doc in hybrid_result["retriever"]["documents"]:
    print(f"  Score: {doc.score:.4f} | {doc.content[:80]}...")

Run it:

python retrieve.py

Expected output (scores will vary slightly):

=== Dense Retrieval ===
  Score: 0.7821 | RAG combines retrieval with generation for grounded LLM answers.
  Score: 0.6543 | Haystack is an open-source LLM framework for building RAG pipelines.
  Score: 0.5912 | Embedding models convert text into dense vectors for semantic search.

=== BM25 Retrieval ===
  Score: 1.8421 | RAG combines retrieval with generation for grounded LLM answers.
  Score: 1.2034 | Haystack is an open-source LLM framework for building RAG pipelines.
  Score: 0.9876 | Embedding models convert text into dense vectors for semantic search.

=== Hybrid Retrieval (alpha=0.5) ===
  Score: 0.8912 | RAG combines retrieval with generation for grounded LLM answers.
  Score: 0.7634 | Haystack is an open-source LLM framework for building RAG pipelines.
  Score: 0.7123 | Embedding models convert text into dense vectors for semantic search.

Understanding alpha in hybrid search

The alpha parameter controls the dense/sparse blend:

• alpha=1.0 → pure dense (vector) search
• alpha=0.0 → pure sparse (BM25) search
• alpha=0.5 → equal weight (default)

Tune this on a validation set. For technical documentation with specific terminology, lower alpha (0.3–0.4) often works better. For conceptual queries, higher alpha (0.6–0.7) wins.

Add metadata filtering

Real workloads need filters — by source, date, version, or custom tags. Update index_docs.py to include metadata:

sample_docs = [
    Document(
        content="Haystack is an open-source LLM framework for building RAG pipelines.",
        meta={"source": "docs", "version": "2.0", "tags": ["framework", "rag"]}
    ),
    Document(
        content="Weaviate is a vector database with hybrid search and GraphQL support.",
        meta={"source": "docs", "version": "1.25", "tags": ["database", "vector"]}
    ),
    Document(
        content="Embedding models convert text into dense vectors for semantic search.",
        meta={"source": "blog", "version": "1.0", "tags": ["embeddings", "ml"]}
    ),
    Document(
        content="RAG combines retrieval with generation for grounded LLM answers.",
        meta={"source": "docs", "version": "2.0", "tags": ["rag", "llm"]}
    ),
    Document(
        content="Batch indexing improves throughput by reducing network round-trips.",
        meta={"source": "blog", "version": "1.0", "tags": ["performance", "indexing"]}
    ),
]

Re-run index_docs.py. Now create filtered_retrieve.py:

from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack_integrations.components.retrievers.weaviate import WeaviateHybridRetriever
from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore
from haystack.dataclasses import Filter

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    similarity="cosine",
)

# Filter: only documents from "docs" source with version "2.0"
filters = Filter(
    operator="AND",
    conditions=[
        {"field": "meta.source", "operator": "==", "value": "docs"},
        {"field": "meta.version", "operator": "==", "value": "2.0"},
    ],
)

pipeline = Pipeline()
pipeline.add_component("embedder", SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"))
pipeline.add_component("retriever", WeaviateHybridRetriever(document_store=document_store, top_k=5, alpha=0.5, filters=filters))
pipeline.connect("embedder.embedding", "retriever.query_embedding")

query = "How does RAG work?"
result = pipeline.run({"embedder": {"text": query}, "retriever": {"query": query}})

print(f"Filtered results (source=docs, version=2.0):")
for doc in result["retriever"]["documents"]:
    print(f"  Score: {doc.score:.4f} | Source: {doc.meta.get('source')} | Version: {doc.meta.get('version')} | {doc.content[:80]}...")

Run it:

python filtered_retrieve.py

Expected output:

Filtered results (source=docs, version=2.0):
  Score: 0.8912 | Source: docs | Version: 2.0 | RAG combines retrieval with generation for grounded LLM answers.
  Score: 0.7634 | Source: docs | Version: 2.0 | Haystack is an open-source LLM framework for building RAG pipelines.

The Filter class supports AND, OR, NOT operators and comparison operators: ==, !=, >, >=, <, <=, in, not in. For array fields like tags, use in:

{"field": "meta.tags", "operator": "in", "value": ["rag"]}

Production considerations

Connection pooling and gRPC

The WeaviateDocumentStore uses gRPC by default (port 50051) for writes and REST for reads. For high-throughput indexing, increase gRPC connection pool size:

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    similarity="cosine",
    # These are passed to the underlying weaviate-client
    additional_config={
        "grpc_compression": "gzip",
        "timeout_config": (5, 60),  # connect, read
    },
)

Authentication

Enable authentication in production. Weaviate supports API keys and OIDC. With API keys:

# docker-compose.yml addition
environment:
  AUTHENTICATION_APIKEY_ENABLED: 'true'
  AUTHENTICATION_APIKEY_ALLOWED_KEYS: 'your-secret-key'
  AUTHENTICATION_APIKEY_USERS: 'your-user'

Then in Haystack:

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    auth_client_secret=("your-user", "your-secret-key"),
)

Monitoring index health

Check object count and vector index status:

# Object count
print(document) count
count = document_store.count_documents()

# Weaviate-specific: get collection stats
collection = document_store._client.collections.get("HaystackDocs")
stats = collection.aggregate.over_all(total_count=True)
print(f"Total objects: {stats.total_count}")

# Check vector index status
config = collection.config.get()
print(f"Vector index type: {config.vector_index_config.vector_index_type}")

Complete RAG pipeline

Wire retrieval into a generator for a minimal RAG loop. Create rag_pipeline.py:

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.generators import OpenAIGenerator
from haystack_integrations.components.retrievers.weaviate import WeaviateHybridRetriever
from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore

document_store = WeaviateDocumentStore(
    host="http://localhost",
    port=8080,
    grpc_port=50051,
    index="HaystackDocs",
    embedding_dim=384,
    similarity="cosine",
)

template = """
Answer the question using only the provided context. If the context doesn't contain the answer, say so.

Context:
{% for doc in documents %}
  {{ doc.content }}
{% endfor %}

Question: {{ query }}
Answer:
"""

rag_pipeline = Pipeline()
rag_pipeline.add_component("embedder", SentenceTransformersTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2"))
rag_pipeline.add_component("retriever", WeaviateHybridRetriever(document_store=document_store, top_k=3, alpha=0.5))
rag_pipeline.add_component("prompt", PromptBuilder(template=template))
rag_pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini"))

rag_pipeline.connect("embedder.embedding", "retriever.query_embedding")
rag_pipeline.connect("retriever.documents", "prompt.documents")
rag_pipeline.connect("prompt", "generator")

query = "What is RAG?"
result = rag_pipeline.run({
    "embedder": {"text": query},
    "retriever": {"query": query},
    "prompt": {"query": query},
})

print(result["generator"]["replies"][0])

Set OPENAI_API_KEY and run:

export OPENAI_API_KEY="sk-..."
python rag_pipeline.py

Expected output (abbreviated):

RAG (Retrieval-Augmented Generation) combines retrieval with generation for grounded LLM answers. It retrieves relevant documents from a knowledge base and uses them as context for the language model to generate accurate, cited responses.

Troubleshooting common issues

Next steps

• Swap SentenceTransformersDocumentEmbedder for a hosted embedder (OpenAI, Cohere, Voyage) by changing the component — no pipeline restructuring needed.
• Add a DocumentSplitter with split_by="sentence" and split_length=2 for longer documents before embedding.
• Implement hybrid search with alpha tuned per query type using a classifier.
• Set up Weaviate backups (docker exec weaviate weaviate backup create ...) before schema migrations.

You now have a working Haystack + Weaviate stack that indexes, filters, and retrieves at production scale. The same patterns apply whether you’re serving 10K or 100M documents — just adjust batch sizes, connection pools, and Weaviate cluster topology.