Engineers who prompt agents clarifying questions often get silence or a flood of irrelevant queries. The fix is to encode clarification triggers directly into the agent’s system prompt and tool schema, then enforce them with a lightweight state machine. This how-to gives you a copy-paste pattern to make your agent ask the right question at the right time.
Step 1: Define the clarification contract
Before writing any prompt, specify exactly which inputs are required for the agent to act safely. Treat the agent like a typed function. If a field is missing, the agent must ask, not guess.
Write a JSON schema that describes the task parameters and marks which are mandatory:
{
"task": "schedule_meeting",
"required": ["attendees", "duration_min", "timezone"],
"optional": ["topic", "preferred_time"],
"constraints": {
"duration_min": "integer > 0",
"attendees": "array of email strings"
}
}Keep this schema in your codebase, not buried in prompt text. You will reference it from both the system prompt and your validation layer.
Step 2: Write a system prompt that mandates clarification
The model will not ask good questions unless you forbid it from proceeding without required fields. Use a strict instruction block. A useful pattern is to force a clarify tool call when any required field is absent.
def build_system_prompt(schema: dict) -> str:
req = ", ".join(schema["required"])
return f"""You are an execution agent for {schema['task']}.
You MUST NOT attempt the task until these required fields are present: {req}.
If any required field is missing, call the `clarify` tool with a single targeted
question for the missing field. Ask only one question per turn. If the user
supplies the field, update state and re-check. Do not ask for optional fields
unless the task is ambiguous without them.
"""This makes the clarification behavior deterministic at the prompt level. The model still decides what to ask, but the contract decides when.
Step 3: Implement a dialogue state checker
Prompting alone is not enough; you need a guard that inspects the model’s output and the user’s replies. A minimal state object tracks which required fields are filled and which questions have been asked.
class ClarificationState:
def __init__(self, schema: dict):
self.required = set(schema["required"])
self.filled = set()
self.asked = set()
def update(self, user_data: dict):
for k in self.required:
if k in user_data and user_data[k]:
self.filled.add(k)
def missing(self):
return self.required - self.filled
def next_question(self):
for field in self.required:
if field not in self.filled and field not in self.asked:
self.asked.add(field)
return field
return NoneCall update after each user message, then next_question to see what the agent should ask. If the model tries to call the task tool while missing() is non-empty, reject it.
Step 4: Run the agent loop with a model API
Wire the pieces together with a standard OpenAI-compatible chat loop. The agent sends the system prompt, the schema, and conversation history. When the state says a field is missing, you inject a forced clarify tool into the model’s available functions.
from openai import OpenAI
client = OpenAI(
base_url="https://api.n4n.ai/v1", # OpenAI-compatible, 240+ models, auto-fallback
api_key="YOUR_KEY"
)
tools = [{
"type": "function",
"function": {
"name": "clarify",
"description": "Ask the user one clarifying question",
"parameters": {
"type": "object",
"properties": {"field": {"type": "string"},
"question": {"type": "string"}},
"required": ["field", "question"]
}
}
}]
def agent_turn(messages, state, schema):
field = state.next_question()
if field:
# Force clarification by only exposing the clarify tool
resp = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=messages,
tools=tools,
tool_choice={"type": "function", "function": {"name": "clarify"}}
)
else:
resp = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=messages
)
return respUsing an OpenAI-compatible gateway such as n4n.ai gives you automatic fallback when a provider is rate-limited or degraded, so the clarification step does not time out under load. The per-token metering also helps you track how many extra turns clarification costs.
Step 5: Verify success with adversarial tests
You cannot ship this blind. Write a test that simulates a vague user and asserts the agent emits a clarify call for the correct field.
def test_agent_asks_for_attendees():
schema = {"task": "schedule_meeting",
"required": ["attendees", "duration_min", "timezone"]}
state = ClarificationState(schema)
messages = [{"role": "user", "content": "Book a meeting."}]
# Simulate state update from empty user payload
state.update({})
field = state.next_question()
assert field == "attendees"
# In real test, mock client.chat.completions to return clarify with fieldRun a suite covering: (1) all fields missing → first missing asked; (2) one field given → next missing asked; (3) all fields given → no clarify tool called. If these pass, your prompt agents clarifying questions logic is sound. Add a regression test where the user replies with gibberish; the state should not mark the field filled.
Step 6: Prevent clarification loops
A naive agent can ask the same question twice if the user replies vaguely. Cap the asks per field at two. After that, escalate to a human or pick a safe default and flag it.
def next_question_capped(self, max_asks=2):
for field in self.required:
if field not in self.filled and self.asked.count(field) < max_asks:
self.asked.append(field)
return field
return NoneLog every clarification turn. In production, you will see which fields users consistently omit; move those to a form UI instead of free text. The goal is to reduce the number of turns needed to reach a fully specified task.
Verify in production
Success means: (a) the agent never executes the task with missing required fields, (b) the average clarifications per session is below 1.5 for returning users, and (c) user satisfaction on ambiguous requests is measurable via follow-up thumbs. Instrument the ClarificationState transitions as events. If you see asked growing without filled, your prompt is not parsing user replies correctly—tighten the schema or add examples to the system prompt.
The pattern above is model-agnostic. Swap the model string for any of the 240+ behind an OpenAI-compatible endpoint and the same guard holds. Prompt agents clarifying questions with a contract, not with hope.