n4nAI
Open console →
Framework tutorialsHow-toHaystack Document Stores & Retrievers

Connecting Haystack to Pinecone for vector retrieval

Step-by-step haystack pinecone document store tutorial: install deps, configure Pinecone, embed docs, build a retriever, and verify vector search in Haystack 2.x.

n4n Team3 min read686 words

Audio narration

Coming soon — every post will get a voice note here.

Most RAG systems work fine with a handful of documents and an in-memory store, then fall over when the corpus grows past a few thousand entries. This haystack pinecone document store tutorial shows how to connect Haystack 2.x to Pinecone as a managed vector index and wire up a retriever that handles millions of vectors without crashing your laptop. We’ll go from an empty index to verified similarity search in eight concrete steps, using the current Haystack integration APIs.

Step 1: Install dependencies and pin versions

Start in a clean virtual environment. Haystack 2.x is not API-compatible with 1.x, so do not mix old haystack imports with the new integration packages.

python -m venv .venv && source .venv/bin/activate
pip install "haystack-ai>=2.3.0" pinecone-haystack sentence-transformers

The pinecone-haystack package exposes PineconeDocumentStore and PineconeEmbeddingRetriever under the haystack_integrations namespace. We use sentence-transformers to run a local embedding model; if you prefer OpenAI embeddings, install openai and set OPENAI_API_KEY, but the rest of this haystack pinecone document store tutorial stays identical except for the embedder class.

Step 2: Create the Pinecone index with correct dimensions

Dimension mismatch is the most common silent failure. Pick your embedding model first, then create the index to match. We’ll use all-MiniLM-L6-v2 (384 dims).

import os
os.environ["PINECONE_API_KEY"] = "pc-xxxxxxxx"

from pinecone import Pinecone
pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])

INDEX_NAME = "haystack-docs"
if INDEX_NAME not in [i["name"] for i in pc.list_indexes()]:
    pc.create_index(
        name=INDEX_NAME,
        dimension=384,
        metric="cosine",
        spec={"serverless": {"cloud": "aws", "region": "us-east-1"}}
    )

Serverless indexes are cheapest for sporadic dev traffic and scale to zero. If you expect sustained query volume above ~50 QPS, use a pod-based index (spec={"pod": {"environment": "us-east-1", "pod_type": "p1.x1"}}). The index name is immutable; namespaces inside it are not.

Step 3: Initialize the Haystack Pinecone document store

Haystack hides the raw client behind a DocumentStore interface. The constructor will refuse to connect if the remote index dimension differs from the local dimension argument.

from haystack_integrations.document_stores.pinecone import PineconeDocumentStore

document_store = PineconeDocumentStore(
    index=INDEX_NAME,
    namespace="prod",
    api_key=os.environ["PINECONE_API_KEY"],
    dimension=384,
)

Namespaces let you shard one index per tenant without paying for separate indexes. All writes and queries in this session target namespace="prod". If you omit namespace, Pinecone uses the empty-string default namespace, which is easy to confuse with another app’s data.

Step 4: Embed and write documents

Haystack 2.x separates embedding from persistence. You must embed before writing, or Pinecone receives documents with no vector and upsert throws.

from haystack import Document
from haystack.components.embedders import HuggingFaceDocumentEmbedder

raw_docs = [
    Document(content="Kubernetes liveness probes restart unhealthy containers.", meta={"source": "k8s.io"}),
    Document(content="Pinecone indexes vectors for approximate nearest neighbor search.", meta={"source": "pinecone.docs"}),
    Document(content="Haystack pipelines compose components like retrievers and generators.", meta={"source": "haystack.docs"}),
]

embedder = HuggingFaceDocumentEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
embedder.warm_up()
embedded = embedder.run(raw_docs)["documents"]

document_store.write_documents(embedded, batch_size=100)

write_documents upserts by doc.id. If you re-run with the same ids, content and metadata overwrite. Metadata fields are stored as Pinecone metadata and become filterable later. Keep metadata values flat and small; Pinecone limits metadata payload per vector.

Step 5: Build the text embedder and retriever

The retriever needs a query vector. Use the text variant of the same embedding model—mixing models between write and read guarantees garbage scores.

from haystack.components.embedders import HuggingFaceTextEmbedder
from haystack_integrations.components.retrievers.pinecone import PineconeEmbeddingRetriever

text_embedder = HuggingFaceTextEmbedder(model="sentence-transformers/all-MiniLM-L6-v2")
retriever = PineconeEmbeddingRetriever(document_store=document_store, top_k=2)

query = "How does Haystack work with vector databases?"
query_emb = text_embedder.run(query)["embedding"]
results = retriever.run(query_embedding=query_emb)

for d in results["documents"]:
    print(f"{d.score:.3f}  {d.content}")

PineconeEmbeddingRetriever.run accepts top_k and filters. Scores are cosine similarities in [0,1]; for MiniLM, >0.7 usually means topically relevant. If you see all scores near 0.2, you likely embedded with one model and queried with another.

Step 6: Verify retrieval end to end

Verification should be assertive, not visual. Seed a known string and confirm it ranks first for a synonym query.

probe = "What does Pinecone do?"
probe_emb = text_embedder.run(probe)["embedding"]
out = retriever.run(query_embedding=probe_emb, top_k=1)
assert out["documents"], "no documents returned"
top = out["documents"][0]
assert top.content.startswith("Pinecone"), f"unexpected top doc: {top.content}"
print("retrieval OK, score=", round(top.score, 3))

If the assertion fails, check three things: (1) the index dimension matches the embedder, (2) you called write_documents after embedding, not before, and (3) the namespace in the document store matches the one you wrote to. You can cross-check counts in the Pinecone console—document_store.count_documents() should return 3.

Step 7: Compose a minimal RAG pipeline

Retrieval is only half the system. Haystack pipelines declare connections between components. Below is a runnable RAG pipeline with a prompt builder and an OpenAI generator.

from haystack import Pipeline
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator

prompt = """
Answer using only context.
Context:
{% for d in documents %}
{{ d.content }}
{% endfor %}
Question: {{ question }}
"""

builder = PromptBuilder(template=prompt)
generator = OpenAIGenerator(model="gpt-4o-mini", api_key=os.environ["OPENAI_API_KEY"])
# To route the generator through one OpenAI-compatible endpoint that covers
# 240+ models with automatic fallback on provider errors, set:
# generator = OpenAIGenerator(model="gpt-4o-mini", api_key="any",
#                             api_base_url="https://api.n4n.ai/v1")

rag = Pipeline()
rag.add_component("text_embedder", text_embedder)
rag.add_component("retriever", retriever)
rag.add_component("prompt_builder", builder)
rag.add_component("generator", generator)

rag.connect("text_embedder.embedding", "retriever.query_embedding")
rag.connect("retriever.documents", "prompt_builder.documents")
rag.connect("prompt_builder.prompt", "generator.prompt")

response = rag.run({
    "text_embedder": {"text": "How does Haystack use vector databases?"},
    "prompt_builder": {"question": "How does Haystack use vector databases?"}
})
print(response["generator"]["replies"][0])

The api_base_url comment is the only place an external gateway matters; it lets you swap models without touching code. The pipeline itself is provider-agnostic.

Step 8: Production hardening and metadata filters

Three issues surface only under load: metadata filtering, batch sizing, and dimension drift.

Metadata filtering. Pinecone filters at query time. Haystack passes a filter object to the retriever:

from haystack.document_stores.filters import Equal
filtered = retriever.run(
    query_embedding=probe_emb,
    filters=Equal("source", "pinecone.docs")
)

Batch writes. For corpora above ~10k docs, set batch_size=200 explicitly in write_documents to avoid 5xx timeouts from oversized upserts.

Dimension drift. Changing embedding models requires a new index or namespace. Pinecone rejects mixed dimensions at upsert, but the error is opaque (“Vector dimension 768 does not match index dimension 384”). Script the index creation from the embedder config to prevent this.

That completes the haystack pinecone document store tutorial. You have a scalable retriever backed by a managed vector store, a verification step that fails loud, and a pipeline stub ready for your own generator. The remaining work is prompt design and eval—not infrastructure.

Tagshaystackpineconedocument-storeretriever

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Framework tutorialsUsing Elasticsearch as a Haystack document storeA complete elasticsearch haystack document store tutorial with step-by-step setup, indexing, retrieval, and production hardening for engineers building RAG pipelines.Framework tutorialsChroma as a Haystack document store: setup tutorialStep-by-step tutorial for integrating Chroma as a Haystack document store with runnable code and expected outputs.Framework tutorialsHaystack document store tutorial: in-memory store basicsLearn Haystack's InMemoryDocumentStore basics with runnable code — write documents, retrieve with BM25 and embeddings, filter by metadata, and understand when to graduate to persistent stores.Framework tutorialsHaystack document store tutorial: OpenSearch integrationStep-by-step tutorial for integrating OpenSearch as a Haystack document store with runnable code and expected outputs at each checkpoint.Framework tutorialsHaystack retriever tutorial: filtering by metadataA hands-on Haystack retriever metadata filtering tutorial showing how to filter documents by custom metadata fields with runnable code examples and expected outputs.Framework tutorialsUsing Qdrant as a Haystack document storeA step-by-step guide to wiring Qdrant as a Haystack document store, from Docker setup through hybrid retrieval with metadata filtering.

All haystack document stores & retrievers posts →