Migrating a raw OpenAI chatbot to CrewAI agents

Migrating a raw OpenAI chatbot to CrewAI means shifting from a single request-response loop to a multi-agent system where specialized agents collaborate on tasks. The primary keyword here is structure: you replace ad-hoc prompt engineering with explicit roles, goals, and task dependencies. This guide walks through the migration end to end with runnable code at each step.

Step 1: Audit your existing OpenAI chatbot

Before writing any CrewAI code, map what your current chatbot actually does. Most raw OpenAI implementations look like this:

# legacy_chatbot.py
from openai import OpenAI

client = OpenAI()

SYSTEM_PROMPT = """You are a helpful assistant that can:
1. Answer general questions
2. Summarize text
3. Extract action items from meeting notes
4. Draft email replies
"""

def chat(user_message: str, history: list[dict] = None) -> str:
    messages = [{"role": "system", "content": SYSTEM_PROMPT}]
    if history:
        messages.extend(history)
    messages.append({"role": "user", "content": user_message})
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        temperature=0.3,
    )
    return response.choices[0].message.content

Identify every distinct capability buried in that system prompt. Each capability becomes a candidate for a dedicated agent. In the example above, four capabilities exist: general Q&A, summarization, action-item extraction, and email drafting. Write them down — this list drives your agent design.

Step 2: Set up the CrewAI environment

Install CrewAI and its dependencies. You’ll also need an LLM provider; CrewAI works with any OpenAI-compatible endpoint.

pip install crewai crewai-tools python-dotenv

Create a minimal config file for your LLM connection. If you’re using an OpenAI-compatible gateway that handles fallback and routing across providers, point base_url there; otherwise use the standard OpenAI endpoint.

# config.py
import os
from crewai import LLM

# Use environment variables for secrets
llm = LLM(
    model="gpt-4o-mini",
    api_key=os.getenv("OPENAI_API_KEY"),
    # base_url="https://api.n4n.ai/v1",  # optional: OpenAI-compatible gateway
    temperature=0.3,
)

Verify the setup works:

# test_llm.py
from config import llm

response = llm.call("Reply with 'OK' only.")
print(response)  # Should print: OK

Run it. If you get a response, the LLM layer is wired correctly.

Step 3: Define agents with roles, goals, and backstories

CrewAI agents are not just prompts — they are persistent personas with explicit responsibilities. Each agent needs a role, a goal, and a backstory that shapes its reasoning style.

# agents.py
from crewai import Agent
from config import llm

general_qa_agent = Agent(
    role="General Knowledge Assistant",
    goal="Answer factual questions accurately and concisely",
    backstory=(
        "You are a reference librarian with broad knowledge across domains. "
        "You prioritize accuracy over creativity and cite uncertainty when needed."
    ),
    llm=llm,
    verbose=True,
    allow_delegation=False,
)

summarizer_agent = Agent(
    role="Content Summarizer",
    goal="Produce faithful, structured summaries of provided text",
    backstory=(
        "You are a technical writer who distills long documents into key points. "
        "You preserve the original meaning without adding interpretation."
    ),
    llm=llm,
    verbose=True,
    allow_delegation=False,
)

action_extractor_agent = Agent(
    role="Action Item Extractor",
    goal="Identify concrete, assigned, time-bound action items from meeting notes",
    backstory=(
        "You are a project manager who reads meeting transcripts and outputs "
        "only actionable tasks with owners and due dates."
    ),
    llm=llm,
    verbose=True,
    allow_delegation=False,
)

email_drafter_agent = Agent(
    role="Professional Email Drafter",
    goal="Draft clear, courteous email replies given context and intent",
    backstory=(
        "You are an executive assistant who writes polished business emails. "
        "You match the sender's tone and include all necessary context."
    ),
    llm=llm,
    verbose=True,
    allow_delegation=False,
)

Key decisions here:

allow_delegation=False keeps agents focused; enable it only if you want agents to hand off sub-tasks dynamically.
verbose=True prints agent reasoning to stdout — essential during migration to verify behavior.
Each agent gets the same LLM instance, but you can assign different models per agent (e.g., a cheaper model for summarization, a stronger one for email drafting).

Step 4: Create tasks that map to your chatbot flows

Tasks define what each agent does, its expected output format, and its dependencies. This is where you replace the monolithic system prompt with structured contracts.

# tasks.py
from crewai import Task
from agents import (
    general_qa_agent,
    summarizer_agent,
    action_extractor_agent,
    email_drafter_agent,
)

# Input placeholders will be filled at runtime via `inputs` dict
general_qa_task = Task(
    description="Answer the user's question: {user_question}",
    expected_output="A concise, accurate answer. If uncertain, say so.",
    agent=general_qa_agent,
)

summarize_task = Task(
    description="Summarize the following text in 5 bullet points:\n{source_text}",
    expected_output="Exactly 5 bullet points covering the main ideas.",
    agent=summarizer_agent,
)

extract_actions_task = Task(
    description=(
        "Extract action items from these meeting notes:\n{meeting_notes}\n\n"
        "Format each as: [Owner] - [Action] - [Due Date or 'TBD']"
    ),
    expected_output="A list of action items, one per line, in the specified format.",
    agent=action_extractor_agent,
)

draft_email_task = Task(
    description=(
        "Draft a professional email reply.\n"
        "Context: {email_context}\n"
        "Intent: {reply_intent}\n"
        "Recipient: {recipient_name}"
    ),
    expected_output="A complete email with greeting, body, and sign-off.",
    agent=email_drafter_agent,
)

Notice the {placeholder} syntax — CrewAI interpolates these from the inputs dictionary you pass at kickoff time. This makes tasks reusable and testable in isolation.

Step 5: Wire up the crew and execution flow

A Crew orchestrates agents and tasks. For a chatbot migration, you typically want a sequential process where a router task classifies the user intent, then the appropriate specialist task runs. CrewAI’s Process.sequential runs tasks in order; Process.hierarchical adds a manager agent that delegates dynamically. Start with sequential — it’s deterministic and easier to debug.

# crew.py
from crewai import Crew, Process
from tasks import (
    general_qa_task,
    summarize_task,
    extract_actions_task,
    draft_email_task,
)
from agents import (
    general_qa_agent,
    summarizer_agent,
    action_extractor_agent,
    email_drafter_agent,
)

# A lightweight router agent to classify intent
router_agent = Agent(
    role="Intent Router",
    goal="Classify the user request into one of: general_qa, summarize, extract_actions, draft_email",
    backstory=(
        "You are a traffic controller. Output only the classification label."
    ),
    llm=general_qa_agent.llm,  # reuse same LLM
    verbose=True,
    allow_delegation=False,
)

router_task = Task(
    description=(
        "Classify this user request: {user_input}\n\n"
        "Categories:\n"
        "1. general_qa — factual questions\n"
        "2. summarize — 'summarize this text: ...'\n"
        "3. extract_actions — 'action items from: ...'\n"
        "4. draft_email — 'draft email to ...'\n\n"
        "Output only the category name."
    ),
    expected_output="One of: general_qa, summarize, extract_actions, draft_email",
    agent=router_agent,
)

# The crew runs router first, then the matched specialist task
# We'll handle the conditional execution in the wrapper (Step 6)
crew = Crew(
    agents=[
        router_agent,
        general_qa_agent,
        summarizer_agent,
        action_extractor_agent,
        email_drafter_agent,
    ],
    tasks=[router_task],  # specialist tasks added dynamically
    process=Process.sequential,
    verbose=True,
)

Step 6: Build the runtime wrapper with state and memory

The raw OpenAI chatbot likely maintained conversation history in a list. CrewAI has two memory systems: short-term memory (conversation history within a crew run) and long-term memory (persisted across runs via memory=True and a storage backend). For a chatbot, you need both: short-term for context within a session, long-term for user preferences across sessions.

# chatbot.py
import json
from typing import Optional
from crew import crew, router_task
from tasks import (
    general_qa_task,
    summarize_task,
    extract_actions_task,
    draft_email_task,
)
from agents import (
    general_qa_agent,
    summarizer_agent,
    action_extractor_agent,
    email_drafter_agent,
)

# Simple in-memory session store (replace with Redis/DB for production)
sessions: dict[str, list[dict]] = {}

INTENT_TO_TASK = {
    "general_qa": (general_qa_task, general_qa_agent),
    "summarize": (summarize_task, summarizer_agent),
    "extract_actions": (extract_actions_task, action_extractor_agent),
    "draft_email": (draft_email_task, email_drafter_agent),
}

def classify_intent(user_input: str) -> str:
    """Run just the router task to get intent."""
    result = crew.kickoff(inputs={"user_input": user_input})
    # CrewAI returns a CrewOutput; the raw string is in .raw
    intent = result.raw.strip().lower()
    return intent if intent in INTENT_TO_TASK else "general_qa"

def run_specialist(intent: str, user_input: str, session_id: str) -> str:
    """Run the appropriate specialist task with session context."""
    task, agent = INTENT_TO_TASK[intent]
    
    # Build inputs: user_input + any session context
    inputs = {"user_input": user_input}
    
    # Add session history as context for the specialist
    history = sessions.get(session_id, [])
    if history:
        # Format last 3 turns as context
        context = "\n".join(
            f"User: {h['user']}\nAssistant: {h['assistant']}" 
            for h in history[-3:]
        )
        inputs["conversation_context"] = context
    
    # Create a temporary crew with just the specialist task
    specialist_crew = Crew(
        agents=[agent],
        tasks=[task],
        process=Process.sequential,
        verbose=True,
    )
    
    result = specialist_crew.kickoff(inputs=inputs)
    return result.raw.strip()

def chat(session_id: str, user_input: str) -> str:
    """Main entry point: classify, execute, store history."""
    intent = classify_intent(user_input)
    response = run_specialist(intent, user_input, session_id)
    
    # Update session history
    if session_id not in sessions:
        sessions[session_id] = []
    sessions[session_id].append({"user": user_input, "assistant": response})
    
    return response

# CLI for manual testing
if __name__ == "__main__":
    sid = "test-session-1"
    print("CrewAI Chatbot ready. Type 'quit' to exit.")
    while True:
        user = input("\nYou: ").strip()
        if user.lower() in ("quit", "exit"):
            break
        try:
            reply = chat(sid, user)
            print(f"\nAssistant: {reply}")
        except Exception as e:
            print(f"\nError: {e}")

Run this and test each capability:

You: What is the capital of Mongolia?
Assistant: Ulaanbaatar is the capital of Mongolia.

You: Summarize this text: CrewAI is a framework for orchestrating...
Assistant: • CrewAI enables multi-agent collaboration...
• Agents have roles, goals, and backstories...
• Tasks define work with expected outputs...
• Crews orchestrate agents via sequential or hierarchical processes...
• Memory supports context across interactions.

You: Action items from: John will finish the report by Friday. Sarah reviews Monday.
Assistant: John - Finish the report - Friday
Sarah - Review the report - Monday

You: Draft email to Alice: confirm meeting at 3pm tomorrow
Assistant: Subject: Meeting Confirmation — Tomorrow at 3 PM

Hi Alice,

Just confirming our meeting tomorrow at 3 PM...

Best regards,
[Your Name]

Step 7: Add observability and error handling

Production migrations need logging, latency tracking, and graceful degradation. Wrap the kickoff calls with a thin observability layer.

# observability.py
import time
import logging
from functools import wraps
from typing import Callable, Any

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("crewai_chatbot")

def observed(func: Callable) -> Callable:
    @wraps(func)
    def wrapper(*args, **kwargs) -> Any:
        start = time.perf_counter()
        try:
            result = func(*args, **kwargs)
            latency_ms = (time.perf_counter() - start) * 1000
            logger.info(f"{func.__name__} succeeded in {latency_ms:.0f}ms")
            return result
        except Exception as e:
            latency_ms = (time.perf_counter() - start) * 1000
            logger.error(f"{func.__name__} failed after {latency_ms:.0f}ms: {e}")
            raise
    return wrapper

Apply it to your core functions:

# chatbot.py (updated imports)
from observability import observed

@observed
def classify_intent(user_input: str) -> str:
    ...

@observed
def run_specialist(intent: str, user_input: str, session_id: str) -> str:
    ...

For rate limits or provider degradation, implement a retry policy with exponential backoff. CrewAI doesn’t include this natively, so wrap the LLM call or use a gateway that handles fallback automatically.

# resilient_llm.py
from crewai import LLM
from tenacity import retry, stop_after_attempt, wait_exponential

class ResilientLLM(LLM):
    @retry(
        wait=wait_exponential(multiplier=1, min=2, max=30),
        stop=stop_after_attempt(3),
    )
    def call(self, messages, **kwargs):
        return super().call(messages, **kwargs)

Then use ResilientLLM in config.py instead of the base LLM.

Step 8: Verify the migration with automated tests

Write tests that exercise each agent path end-to-end. Use a deterministic model (temperature=0) or mock the LLM for unit tests.

# test_chatbot.py
import pytest
from chatbot import chat, classify_intent, run_specialist, sessions

@pytest.fixture(autouse=True)
def clear_sessions():
    sessions.clear()
    yield
    sessions.clear()

def test_general_qa():
    reply = chat("test-1", "What is 2+2?")
    assert "4" in reply

def test_summarize():
    text = "CrewAI lets you build multi-agent systems. Agents have roles. Tasks define work."
    reply = chat("test-2", f"Summarize this text: {text}")
    assert "•" in reply or "-" in reply  # bullet points

def test_extract_actions():
    notes = "Alice will deploy by Friday. Bob reviews Monday."
    reply = chat("test-3", f"Action items from: {notes}")
    assert "Alice" in reply and "Friday" in reply
    assert "Bob" in reply and "Monday" in reply

def test_draft_email():
    reply = chat("test-4", "Draft email to Carol: confirm Tuesday 10am")
    assert "Carol" in reply or "Tuesday" in reply
    assert "10" in reply or "10am" in reply.lower()

def test_session_memory():
    chat("test-5", "My name is Dave.")
    reply = chat("test-5", "What is my name?")
    assert "Dave" in reply

def test_intent_classification():
    assert classify_intent("What is Python?") == "general_qa"
    assert classify_intent("Summarize this: hello world") == "summarize"
    assert classify_intent("Action items from: meeting notes") == "extract_actions"
    assert classify_intent("Draft email to Bob") == "draft_email"

Run with pytest -v. All tests should pass. If any fail, the verbose=True output from CrewAI will show the agent’s reasoning trace — use it to refine backstories or expected output formats.

Step 9: Deploy and monitor

Containerize the service. A minimal Dockerfile:

# Dockerfile
FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENV PYTHONUNBUFFERED=1
CMD ["python", "chatbot.py"]

# requirements.txt
crewai==0.80.0
crewai-tools==0.1.0
python-dotenv==1.0.1
tenacity==8.3.0

Deploy behind a load balancer. Expose a /chat endpoint (FastAPI, Flask, or your framework of choice) that calls chat(session_id, user_input). Emit structured logs (JSON) for your observability stack — include session_id, intent, latency_ms, token_usage (available via crew.usage_metrics after kickoff), and success boolean.

Set up alerts on:

P95 latency > 10s (indicates provider degradation)
Error rate > 1% (indicates prompt or parsing failures)
Token usage spikes (indicates runaway loops or prompt injection)

What changes after migration

Aspect	Raw OpenAI	CrewAI
Prompt management	One monolithic system prompt	Distributed across agent backstories + task descriptions
Control flow	Implicit in code	Explicit in task dependencies and crew process
State	Manual history list	Short-term (session) + long-term (persistent) memory
Observability	Custom logging	Built-in verbose traces + usage metrics
Extensibility	Edit prompt, redeploy	Add agent/task, wire into crew
Debugging	Print messages	Read agent reasoning logs

The migration pays off when you need to add a fifth capability — say, code generation. You create a CodeAgent and CodeTask, register them in INTENT_TO_TASK, and the router (or a hierarchical manager) starts delegating automatically. No prompt surgery required.

Start small. Migrate one capability at a time, verify with the test suite, then cut over traffic. The crew abstraction makes rollback trivial: keep the old chat() function alongside the new one, feature-flag the entry point, and compare outputs side by side.