LlamaIndex agents vs LangChain agents compared

If you’re evaluating the llamaindex vs langchain agents comparison for a production system, the decision rarely comes down to “which is better.” It comes down to whether your workload centers on structured data retrieval with light reasoning, or open-ended multi-step workflows that need flexible composition. Both frameworks have converged on similar surfaces — OpenAI function calling, streaming, async — but their primitives, opinions, and escape hatches differ in ways that show up in latency, debuggability, and maintenance burden.

Core architecture differences

LlamaIndex began as a data framework (GPT Index) for retrieval-augmented generation. Its agent layer sits on top of query engines, retrievers, and response synthesizers — primitives designed for structured data access. An agent in LlamaIndex is essentially a QueryEngineTool wrapped in an AgentWorker that decides which tool to invoke and how to synthesize results. The framework assumes you’re querying indexes, SQL databases, or APIs that return structured payloads.

LangChain started as a composition framework. Its core abstraction is the Runnable interface (LCEL — LangChain Expression Language), and agents are just Runnables that happen to invoke tools in a loop. There’s no privileged “query engine” concept; a tool is any function with a JSON schema. This makes LangChain agents more generic but also less opinionated about how data flows from source to answer.

# LlamaIndex: agent built around query engines
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import QueryEngineTool

query_engine = index.as_query_engine(similarity_top_k=5)
tool = QueryEngineTool.from_defaults(
    query_engine=query_engine,
    name="sec_filings",
    description="Search SEC 10-K filings for financial data"
)
agent = ReActAgent.from_tools([tool], llm=llm, verbose=True)

# LangChain: agent built around arbitrary tools
from langchain.agents import create_openai_functions_agent, AgentExecutor
from langchain.tools import tool

@tool
def sec_filings_search(query: str) -> str:
    """Search SEC 10-K filings for financial data"""
    return retrieve_from_vectorstore(query)

agent = create_openai_functions_agent(llm, [sec_filings_search], prompt)
executor = AgentExecutor(agent=agent, tools=[sec_filings_search], verbose=True)

The practical difference: LlamaIndex agents excel when your tools are query engines over indexed data. LangChain agents excel when your tools are heterogeneous — browser, code interpreter, API clients, custom logic — and you need fine-grained control over the reasoning loop.

Agent abstractions and patterns

LlamaIndex ships with a small set of agent implementations: ReActAgent, OpenAIAgent (function-calling native), and FunctionCallingAgentWorker. The AgentWorker base class exposes initialize_step, run_step, and finalize_step hooks, giving you insertion points for logging, guardrails, or custom step logic. But the repertoire is intentionally narrow — the framework steers you toward ReAct or OpenAI function calling.

LangChain offers more agent types out of the box: create_react_agent, create_openai_functions_agent, create_xml_agent, create_json_agent, create_structured_chat_agent, plus the newer create_tool_calling_agent that works across any model supporting tool use. Each maps to a different prompt strategy and parsing logic. You can also build a custom agent by implementing the Runnable interface directly — a common pattern when you need non-standard control flow (e.g., human-in-the-loop, parallel tool execution, dynamic tool selection).

# LangChain: custom agent via Runnable for parallel tool calls
from langchain_core.runnables import RunnableLambda, RunnableParallel

def select_tools(state):
    # custom logic: pick 2-3 tools to run in parallel
    return {"tool_calls": choose_tools(state["input"])}

parallel_executor = RunnableParallel(
    tool1=tool1, tool2=tool2, tool3=tool3
)

custom_agent = (
    RunnableLambda(select_tools)
    | parallel_executor
    | RunnableLambda(synthesize_results)
)

LlamaIndex’s AgentWorker can achieve similar patterns but requires subclassing and overriding run_step — more ceremony, less composition.

Tool use and function calling

Both frameworks now default to OpenAI-style function calling when the model supports it, falling back to ReAct prompting otherwise. The divergence is in tool definition ergonomics and argument validation.

LlamaIndex tools inherit from BaseTool or FunctionTool. FunctionTool.from_defaults(fn=my_func) infers the JSON schema from type hints and docstrings — clean for Python functions. QueryEngineTool and RetrieverTool are specialized subclasses that handle the query-engine-to-tool bridging automatically. Argument validation happens at call time via Pydantic.

LangChain tools use the @tool decorator or StructuredTool.from_function. The decorator infers schema similarly, but LangChain also supports args_schema: Type[BaseModel] for explicit Pydantic models — useful when the function signature doesn’t match the desired schema (e.g., optional defaults, nested objects). LangChain’s ToolCallable protocol is more permissive, accepting callables, coroutines, or Runnables.

# LlamaIndex: explicit schema via FunctionTool
from llama_index.core.tools import FunctionTool
from pydantic import BaseModel, Field

class SearchArgs(BaseModel):
    query: str = Field(description="Search query")
    top_k: int = Field(default=5, ge=1, le=20)

def search_fn(query: str, top_k: int = 5) -> str:
    return vector_store.query(query, top_k)

tool = FunctionTool.from_defaults(
    fn=search_fn,
    name="vector_search",
    description="Search the vector store",
    fn_schema=SearchArgs  # optional but recommended
)

# LangChain: explicit schema via args_schema
from langchain.tools import StructuredTool
from pydantic import BaseModel, Field

class SearchArgs(BaseModel):
    query: str = Field(description="Search query")
    top_k: int = Field(default=5, ge=1, le=20)

def search_fn(query: str, top_k: int = 5) -> str:
    return vector_store.query(query, top_k)

tool = StructuredTool.from_function(
    func=search_fn,
    name="vector_search",
    description="Search the vector store",
    args_schema=SearchArgs
)

In practice, LlamaIndex’s tool system feels tighter for data-centric tools; LangChain’s feels more flexible for arbitrary side effects (writing files, triggering webhooks, spawning subprocesses).

Memory and state management

LlamaIndex provides ChatMemoryBuffer — a token-limited buffer that summarizes older messages when the limit is exceeded. It integrates with the agent via memory=ChatMemoryBuffer.from_defaults(token_limit=3000). There’s also VectorStoreMemory for semantic retrieval over history, but it’s less mature. State across agent runs is not a first-class concept; you manage it externally (e.g., pass chat_history to agent.chat()).

LangChain has a richer memory taxonomy: ConversationBufferMemory, ConversationBufferWindowMemory, ConversationSummaryMemory, ConversationSummaryBufferMemory, ConversationKGMemory (knowledge graph), and VectorStoreRetrieverMemory. Each implements load_memory_variables and save_context, and they plug into chains/agents via the memory kwarg. LangChain also introduced RunnableWithMessageHistory — a wrapper that automatically fetches and persists history per session ID, backed by any BaseChatMessageHistory implementation (Redis, Postgres, DynamoDB, in-memory).

# LangChain: persistent per-session history with Redis
from langchain_community.chat_message_histories import RedisChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

def get_history(session_id: str) -> RedisChatMessageHistory:
    return RedisChatMessageHistory(session_id, url=redis_url)

agent_with_history = RunnableWithMessageHistory(
    executor,
    get_history,
    input_messages_key="input",
    history_messages_key="chat_history"
)

# invoke with session_id
agent_with_history.invoke({"input": "..."}, config={"configurable": {"session_id": "user-123"}})

LlamaIndex has no direct equivalent — you’d build the session lookup and history injection yourself. For multi-tenant or long-running conversational agents, LangChain’s memory abstractions save significant boilerplate.

Evaluation and observability

LlamaIndex bakes evaluation into the core library: FaithfulnessEvaluator, RelevancyEvaluator, CorrectnessEvaluator, SemanticSimilarityEvaluator, PairwiseComparisonEvaluator, and BatchEvalRunner for CI/CD integration. These work against any query engine or agent response, using an LLM-as-judge pattern. The API is synchronous and straightforward:

from llama_index.core.evaluation import FaithfulnessEvaluator, RelevancyEvaluator

faithfulness = FaithfulnessEvaluator(llm=eval_llm)
relevancy = RelevancyEvaluator(llm=eval_llm)

result = faithfulness.evaluate_response(response=agent_response)
print(result.passing, result.feedback, result.score)

LangChain delegates evaluation to LangSmith (hosted) or the open-source langchain.evaluation module (criteria evaluators, embedding distance, QA evaluators). The local evaluators are functional but less comprehensive than LlamaIndex’s built-ins. LangSmith adds tracing, datasets, regression testing, and a UI — valuable for teams, but a separate dependency and potential vendor lock-in.

For pure local evaluation without a SaaS dependency, LlamaIndex wins on breadth. For team workflows with trace-driven debugging, LangSmith is a genuine differentiator.

Ecosystem and integrations

LangChain’s integration surface is larger — 600+ packages across langchain-community, langchain-* partner packages, and community contributions. If a vector store, document loader, or API wrapper exists, there’s likely a LangChain integration. The trade-off: quality varies, version churn is high, and langchain-community is a frequent source of dependency conflicts.

LlamaIndex’s ecosystem is smaller but more curated. Core integrations (vector stores, LLMs, embedding models, readers) live in llama-index-core or first-party llama-index-* packages. The LlamaHub registry indexes loaders and tools, but the total count is lower. LlamaIndex’s LlamaParse (PDF parsing with layout awareness) and LlamaCloud (managed indexing) are proprietary but technically differentiated — no direct LangChain equivalent.

Both frameworks support custom LLM wrappers. If you’re routing through an inference gateway that exposes an OpenAI-compatible endpoint with automatic fallback and per-token metering, both will work with minimal configuration — just point the base URL and pass headers.

Comparison table

Which to choose

Choose LlamaIndex agents when:

• Your agent’s primary job is querying indexed data (vector, keyword, SQL, graph) and synthesizing answers. The query engine → tool → agent pipeline is purpose-built for this.
• You want batteries-included evaluation without a SaaS dependency. The local evaluators cover RAG quality dimensions out of the box.
• You prefer fewer moving parts and a lighter dependency graph. Core + one LLM package + one vector store package gets you far.
• You need LlamaParse for complex PDF/table extraction and want it integrated natively.
• Your team is smaller and values opinionated defaults over configurability.

Choose LangChain agents when:

• Your tools are heterogeneous — browser, code interpreter, shell, REST APIs, database writers, custom microservices — not just query engines.
• You need fine-grained control over the reasoning loop: parallel tool execution, dynamic tool selection, human-in-the-loop checkpoints, custom parsing logic.
• You require production-grade session memory with pluggable backends (Redis, Postgres, DynamoDB) and automatic history injection per user/session.
• Your team invests in LangSmith for tracing, dataset curation, and regression testing across prompt/model changes.
• You’re building multi-agent systems (supervisor + workers, swarm patterns) where LCEL’s composition model shines.

Choose neither (or both) when:

• You’re building a single-turn RAG endpoint — a thin FastAPI route calling a retriever + LLM is simpler, faster, and easier to observe than either framework’s agent overhead.
• You need deterministic, low-latency tool chains — consider a lightweight orchestration layer (e.g., pydantic-ai, instructor, or plain asyncio + function calling) instead of an agent loop.
• Your latency budget is < 500ms p99 — agent loops add variable round-trips. Profile before committing.

The frameworks aren’t mutually exclusive. A common pattern: LlamaIndex for the data plane (indexing, retrieval, query engines) exposed as tools, consumed by a LangChain agent that orchestrates across data and non-data actions. The integration cost is one FunctionTool wrapper per query engine.