Migrating the raw OpenAI SDK to LangChain step by step

You built a working prototype with the raw OpenAI Python client, but prompt templating and retries are getting messy. This guide shows how to migrate openai sdk to langchain without throwing away your existing call logic. We’ll swap the client, wrap prompts, and keep your tests green.

Step 1: Audit your existing OpenAI SDK calls

Before changing anything, map every openai.ChatCompletion.create (pre-v1) or client.chat.completions.create (v1+) invocation in your codebase. Note the model name, temperature, max_tokens, and the exact message list. A typical raw v1 call looks like this:

from openai import OpenAI
client = OpenAI(api_key="sk-...")

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a concise translator."},
        {"role": "user", "content": "Translate to French: hello world"}
    ],
    temperature=0.2,
)
print(resp.choices[0].message.content)

Identify hardcoded prompts, any post-processing (JSON parsing, regex extraction), and error handling. LangChain won’t magically fix messy output handling, so keep those functions intact. The goal when you migrate openai sdk to langchain is to replace the transport and message construction, not the business logic.

Step 2: Install LangChain and instantiate ChatOpenAI

Install the OpenAI integration package. As of LangChain 0.2+, the chat model lives in langchain-openai, separate from the core package.

pip install langchain-openai

Replace the OpenAI client with ChatOpenAI. The constructor mirrors most parameters you already use:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0.2,
    api_key="sk-...",
)

If you need to migrate openai sdk to langchain gradually, you can keep both clients in the same module and switch one function at a time. Set the OPENAI_API_KEY environment variable instead of hardcoding; ChatOpenAI reads it automatically.

Step 3: Convert message dicts to LangChain messages

LangChain uses SystemMessage, HumanMessage, and AIMessage objects instead of plain dicts. Rewrite the call:

from langchain_core.messages import SystemMessage, HumanMessage

result = llm.invoke([
    SystemMessage(content="You are a concise translator."),
    HumanMessage(content="Translate to French: hello world"),
])
print(result.content)

For batch inference use llm.batch([...]); for async use await llm.ainvoke([...]). The response object exposes .content directly, dropping the .choices[0].message nesting. If you previously accessed resp.usage, LangChain returns result.response_metadata["usage"] (structure varies by provider).

Step 4: Externalize prompts with PromptTemplate

Hardcoded strings block reuse and testing. Pull the user text into a ChatPromptTemplate:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a concise translator."),
    ("user", "Translate to {target_lang}: {text}")
])

chain = prompt | llm
out = chain.invoke({"target_lang": "French", "text": "hello world"})
print(out.content)

The pipe operator (|) is the core LangChain primitive: it composes a prompt, a model, and an optional parser into a runnable. When you migrate openai sdk to langchain, treat every former messages=[...] block as a candidate for ChatPromptTemplate. You can add few-shot examples by mixing ("human", ...) and ("ai", ...) tuples.

Step 5: Compose multi-step chains

Raw SDK code often hand-rolls loops: call model, parse, call again. LangChain expresses that as RunnableSequence or RunnableParallel. Example: translate then summarize.

from langchain_core.output_parsers import StrOutputParser

translate_chain = ChatPromptTemplate.from_messages([
    ("system", "Translate to {lang}."),
    ("user", "{text}")
]) | llm | StrOutputParser()

summarize_chain = ChatPromptTemplate.from_messages([
    ("system", "Summarize in one sentence."),
    ("user", "{translated}")
]) | llm | StrOutputParser()

full = translate_chain | (lambda x: {"translated": x}) | summarize_chain
print(full.invoke({"lang": "French", "text": "Long English paragraph..."}))

The StrOutputParser removes the need to touch .content manually. Add with_retry() to any runnable for exponential backoff:

chain = (prompt | llm).with_retry(stop_after_attempt=3)

Step 6: Preserve streaming and async behavior

If your original SDK used stream=True, LangChain supports .stream():

for chunk in chain.stream({"target_lang": "French", "text": "hello"}):
    print(chunk.content, end="", flush=True)

Async code maps to ainvoke/astream. Don’t wrap LangChain calls in asyncio.run inside a running loop; use await directly in your FastAPI or asyncio app. Callbacks (callbacks=[...]) let you replicate any logging or token-cost tracking you had in the raw SDK.

Step 7: Point to an OpenAI-compatible gateway (optional)

If you want automatic fallback across providers without code changes, set base_url to an OpenAI-compatible endpoint. For example, n4n.ai exposes one endpoint covering 240+ models and honors client routing directives, so a single ChatOpenAI(base_url="https://api.n4n.ai/v1", api_key=...) gains fallback when a provider is degraded. Your LangChain code stays identical; only the constructor changes.

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0.2,
    base_url="https://api.n4n.ai/v1",
    api_key=os.environ["N4N_KEY"],
)

This is the cleanest way to migrate openai sdk to langchain while gaining resilience and per-token metering without custom retry logic.

Step 8: Verify success with contract tests

Don’t trust the migration until outputs match. Write a small pytest that checks structure and (where the model supports seed) determinism:

def test_translation_shape():
    out = chain.invoke({"target_lang": "French", "text": "hello"})
    assert isinstance(out.content, str)
    assert len(out.content) > 0

Run against a stub or record/replay HTTP (e.g., vcrpy) to avoid burning tokens in CI. If you swapped base_url to a gateway, check the usage metering headers to confirm per-token billing is flowing. Keep the old raw SDK path behind a feature flag for one release as a fallback.

Common pitfalls

• Message order: LangChain does not reorder; system must be first or the API will reject.
• Temperature defaults: LangChain’s default is 0.7; explicitly set it to match your prior behavior.
• JSON mode: Use model_kwargs={"response_format": {"type": "json_object"}} on ChatOpenAI if you relied on SDK JSON mode.
• Token counting: response_metadata is provider-specific; write a thin adapter if you log usage.

Migrating is mechanical once prompts are templated. The payoff is composability, not just cleaner calls.