Semantic Kernel planner tutorial: debugging failed plans

When a Semantic Kernel planner fails, the error surface is rarely the planner itself — it’s usually a mismatch between function signatures, schema validation, or the model’s ability to reason about available tools. This guide walks through a repeatable debugging path for semantic kernel planner debugging failed plans, ordered from fastest checks to deepest inspection.

Start with the plan representation

Before touching the model, inspect what the planner actually produced. The Plan object exposes the serialized steps, and most failures are visible here without a single LLM call.

from semantic_kernel.planners import FunctionCallingStepwisePlanner
from semantic_kernel import Kernel

kernel = Kernel()
# ... add services, import plugins ...

planner = FunctionCallingStepwisePlanner(
    service_id="default",
    max_iterations=10,
    max_tokens=4000,
)

plan = await planner.create_plan(goal="Summarize the last 5 emails and send a digest to the team")
print(plan.generated_plan)

Look for three specific failure modes in the output:

1. Missing function references — steps reference functions that don’t exist in the kernel’s plugin collection
2. Argument type mismatches — the planner passes a string where the function expects a complex object
3. Circular or impossible dependencies — step B requires output from step A, but step A never produces it

The generated_plan property returns a list of FunctionCallingStep objects. Each step has name, arguments, and output_variable. Dump these to JSON and diff against your registered functions.

Validate function schemas against the planner’s expectations

Semantic Kernel planners rely on the kernel’s function metadata — name, description, parameters, and return type. If any of these drift from what the model expects, the plan fails at execution time, not planning time.

Run this validation check after registering plugins:

from semantic_kernel.functions import KernelFunctionMetadata

def validate_plugin_schemas(kernel: Kernel) -> list[str]:
    issues = []
    for plugin_name, plugin in kernel.plugins.items():
        for func_name, func in plugin.functions.items():
            meta: KernelFunctionMetadata = func.metadata
            # Description must be non-empty and specific
            if not meta.description or len(meta.description) < 20:
                issues.append(f"{plugin_name}.{func_name}: description too short or missing")
            # Parameters must have types and descriptions
            for param in meta.parameters:
                if not param.description:
                    issues.append(f"{plugin_name}.{func_name}.{param.name}: missing parameter description")
                if param.schema_data.get("type") is None:
                    issues.append(f"{plugin_name}.{func_name}.{param.name}: missing type in schema")
    return issues

issues = validate_plugin_schemas(kernel)
for issue in issues:
    print(f"SCHEMA ISSUE: {issue}")

The planner uses these descriptions to decide which function to call and how to populate arguments. Vague descriptions like “Gets data” cause the model to hallucinate parameters. Write descriptions as if explaining to a junior engineer: “Retrieves the last N emails from the authenticated user’s inbox. Returns a list of EmailSummary objects with subject, sender, and snippet fields.”

Enable structured logging for the planning loop

The stepwise planner makes multiple model calls — one per iteration. Each call includes the current plan state, available functions, and the goal. Log the full request/response payload to see where reasoning breaks down.

import logging
import json
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

# Configure structured logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("semantic_kernel.planners.function_calling_stepwise_planner")
logger.setLevel(logging.DEBUG)

# Or capture at the service level
class DebuggingChatCompletion(OpenAIChatCompletion):
    async def get_chat_message_contents(self, *args, **kwargs):
        request_messages = kwargs.get("messages", [])
        logger.debug("PLANNER REQUEST:\n%s", json.dumps([m.to_dict() for m in request_messages], indent=2))
        result = await super().get_chat_message_contents(*args, **kwargs)
        logger.debug("PLANNER RESPONSE:\n%s", json.dumps([m.to_dict() for m in result], indent=2))
        return result

# Use the debugging wrapper
kernel.add_service(DebuggingChatCompletion(service_id="default", ...))

Look for these patterns in the logs:

• Function calling format errors — the model emits malformed tool calls (wrong JSON, missing required fields)
• Hallucinated functions — the model calls functions that don’t exist in the provided schema
• Premature termination — the model returns a final answer without calling needed functions
• Argument fabrication — the model invents parameter values not grounded in prior step outputs

Isolate the planner from your business logic

Create a minimal reproduction that exercises only the planner and the specific functions involved in the failure. Strip away authentication, database calls, and external APIs. Replace them with stubs that return deterministic data matching the expected schema.

from semantic_kernel.functions import kernel_function
from pydantic import BaseModel

class EmailSummary(BaseModel):
    subject: str
    sender: str
    snippet: str

class StubEmailPlugin:
    @kernel_function(
        name="get_recent_emails",
        description="Retrieves the last N emails from the authenticated user's inbox"
    )
    async def get_recent_emails(self, count: int = 5) -> list[EmailSummary]:
        return [
            EmailSummary(subject=f"Email {i}", sender=f"user{i}@example.com", snippet=f"Snippet {i}")
            for i in range(count)
        ]

    @kernel_function(
        name="send_digest",
        description="Sends a digest email to the specified recipients with the given content"
    )
    async def send_digest(self, recipients: list[str], content: str) -> str:
        return f"Digest sent to {', '.join(recipients)}"

# Register only the stub
kernel.add_plugin(StubEmailPlugin(), plugin_name="email")

If the plan succeeds with stubs but fails with real implementations, the issue is in your function implementations — not the planner. Common culprits:

• Functions throwing exceptions that aren’t caught and surfaced as function results
• Functions returning types that don’t match the declared return annotation
• Async functions not awaited properly in the plugin

Inspect the function calling model’s raw output

When the planner fails, the underlying model often produces valid JSON that the parser rejects due to schema mismatches. Capture the raw model output before SK’s function calling parser processes it.

from semantic_kernel.connectors.ai.function_calling_utils import parse_function_calls
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion

class RawCaptureChatCompletion(OpenAIChatCompletion):
    async def get_chat_message_contents(self, *args, **kwargs):
        result = await super().get_chat_message_contents(*args, **kwargs)
        for msg in result:
            if msg.items:
                for item in msg.items:
                    if hasattr(item, 'function_call'):
                        # This is the raw function call before validation
                        logger.debug("RAW FUNCTION CALL: %s", item.function_call)
        return result

Compare the raw function call against the function’s KernelFunctionMetadata.parameters. The planner passes arguments as a JSON object. If your function expects count: int but the model passes "count": "5" (string), the kernel will reject it. This is the most common type coercion failure.

Handle the “no valid function” fallback

When the planner cannot find a suitable function, it may return a final answer without calling tools — or it may loop until max_iterations expires. Both look like success to the caller but produce wrong results.

Add a plan validator that runs after planning completes:

from semantic_kernel.planners import FunctionCallingStepwisePlanner
from semantic_kernel import KernelArguments

async def execute_with_validation(kernel: Kernel, goal: str) -> str:
    planner = FunctionCallingStepwisePlanner(
        service_id="default",
        max_iterations=10,
    )
    
    plan = await planner.create_plan(goal)
    
    # Validate: plan must contain at least one function call step
    function_steps = [s for s in plan.generated_plan if s.name != "final_answer"]
    if not function_steps:
        raise ValueError(f"Planner produced no function calls for goal: {goal}")
    
    # Validate: all referenced functions exist
    available_functions = set()
    for plugin in kernel.plugins.values():
        for func in plugin.functions.values():
            available_functions.add(f"{plugin.name}.{func.name}")
    
    for step in function_steps:
        if step.name not in available_functions:
            raise ValueError(f"Plan references unknown function: {step.name}")
    
    # Execute
    result = await planner.execute_plan(plan, kernel)
    return str(result)

This catches the silent failure mode where the model “gives up” and returns a conversational answer instead of executing tools.

Debug argument binding with a custom invoker

The kernel’s function invocation pipeline binds arguments from the plan to the function signature. When this fails, you get a generic error. Wrap the invoker to see exactly what arguments arrive at each function.

from semantic_kernel.functions import KernelFunction
from semantic_kernel import KernelArguments

class DebuggingKernelFunction(KernelFunction):
    def __init__(self, inner: KernelFunction):
        self._inner = inner
        super().__init__(
            name=inner.name,
            plugin_name=inner.plugin_name,
            description=inner.description,
            parameters=inner.parameters,
            is_asynchronous=inner.is_asynchronous,
            return_parameter=inner.return_parameter,
        )
    
    async def invoke(self, kernel: Kernel, arguments: KernelArguments, **kwargs):
        logger.debug("INVOKING %s.%s with args: %s", self.plugin_name, self.name, arguments)
        try:
            result = await self._inner.invoke(kernel, arguments, **kwargs)
            logger.debug("RESULT: %s", result)
            return result
        except Exception as e:
            logger.exception("INVOCATION FAILED for %s.%s", self.plugin_name, self.name)
            raise

# Wrap all functions after plugin registration
for plugin_name, plugin in kernel.plugins.items():
    for func_name, func in plugin.functions.items():
        plugin.functions[func_name] = DebuggingKernelFunction(func)

This reveals binding failures like:

• Missing required arguments the planner didn’t provide
• Extra arguments the function doesn’t accept
• Type mismatches during Pydantic model construction

Reduce planner complexity with explicit function selection

The stepwise planner’s strength — autonomous multi-step reasoning — is also its weakness. For production workflows, consider replacing the planner with explicit function chaining where the sequence is deterministic.

# Instead of planner for "summarize emails and send digest"
async def summarize_and_send_digest(kernel: Kernel, count: int = 5, recipients: list[str] = None):
    email_plugin = kernel.plugins["email"]
    
    # Step 1: Get emails (deterministic)
    emails_result = await email_plugin["get_recent_emails"].invoke(kernel, KernelArguments(count=count))
    emails = emails_result.value
    
    # Step 2: Summarize (single LLM call, no planner)
    summarizer = kernel.get_service("default")
    summary_prompt = f"Summarize these emails in 3 bullets:\n{emails}"
    summary = await summarizer.get_chat_message_contents(summary_prompt)
    
    # Step 3: Send digest (deterministic)
    await email_plugin["send_digest"].invoke(
        kernel, 
        KernelArguments(recipients=recipients or ["team@example.com"], content=str(summary))
    )

Use the planner only for genuinely open-ended goals where the step sequence cannot be predetermined. For everything else, explicit code is more debuggable, testable, and performant.

Common pitfalls and their fixes

When to escalate to model capabilities

If you’ve validated schemas, logged raw outputs, stubbed dependencies, and the planner still produces invalid plans — the model may lack the reasoning capacity for your task complexity. Test the same goal with a more capable model (GPT-4o, Claude 3.5 Sonnet) before concluding the planner is broken.

You can route different goals to different models through the kernel’s service selection:

# Simple goals -> cheaper/faster model
simple_kernel = Kernel()
simple_kernel.add_service(OpenAIChatCompletion(service_id="planner", model_id="gpt-4o-mini"))

# Complex goals -> more capable model
complex_kernel = Kernel()
complex_kernel.add_service(OpenAIChatCompletion(service_id="planner", model_id="gpt-4o"))

# Select at runtime
kernel = complex_kernel if is_complex_goal(goal) else simple_kernel

This is where a gateway like n4n.ai helps — one endpoint, automatic fallback when a provider degrades, and per-token metering so you can measure the cost of planner retries across models.

Summary checklist

Before filing an issue or rewriting your planner integration, run through this sequence:

1. Dump plan.generated_plan and verify every function exists in the kernel
2. Run schema validation on all plugin functions — descriptions, types, required fields
3. Enable debug logging on the chat completion service; capture one full planning loop
4. Replace external dependencies with deterministic stubs matching declared schemas
5. Inspect raw function calls before SK’s parser validates them
6. Add plan validation: at least one function call, no unknown functions
7. Wrap function invocation to log bound arguments and results
8. If the plan is deterministic, replace the planner with explicit function chaining

Most “planner bugs” are schema bugs, description bugs, or model capability mismatches. The planner itself is a thin reasoning loop — the work is in the metadata you feed it.