Designing a reflection loop that knows when to stop

An agent reflection loop lets a model critique and revise its own output, but unbounded iteration burns tokens and latency without improving quality. The hard part is not triggering reflection—it’s knowing when the loop should terminate. This guide gives an ordered path to build a stop-aware agent reflection loop that you can ship without guessing.

1. Define termination before you write the loop

Treat stop conditions as first-class requirements, not afterthoughts. A loop needs at least one hard ceiling (max iterations or timeout) and one quality signal (acceptable score or no meaningful delta). Without both, you will watch a model rephrase the same sentence ten times while your p95 latency climbs.

Write the contract first:

from dataclasses import dataclass

@dataclass
class LoopConfig:
    max_iterations: int = 5
    min_score: float = 0.8
    min_delta: float = 0.02   # absolute score change below this = stagnation
    timeout_ms: int = 30000
    token_budget: int = 8000  # hard cap on total tokens across steps

If the loop hits max_iterations, timeout_ms, or token_budget, it returns the best draft seen so far and flags converged=False. Never let the caller assume success. The agent reflection loop is a local optimizer, not a theorem prover.

2. Separate generation from critique

Coupling the writer and the critic in one prompt invites lazy self-approval. Split them. The generator produces a candidate; the critic evaluates against a fixed rubric. The loop then decides whether to revise.

def run_loop(task: str, cfg: LoopConfig):
    draft = generator.complete(task)
    best_draft, best_score = draft, 0.0
    prev_score = None
    stale = 0

    for i in range(cfg.max_iterations):
        critique = critic.evaluate(task, draft)
        if not critique.valid:
            critique.score = 0.0  # fail safe
        if critique.score > best_score:
            best_draft, best_score = draft, critique.score
        if critique.score >= cfg.min_score:
            return best_draft, True, i, best_score
        if prev_score is not None and abs(critique.score - prev_score) < cfg.min_delta:
            stale += 1
            if stale >= 2:
                return best_draft, False, i, best_score
        else:
            stale = 0
        draft = generator.revise(task, draft, critique)
        prev_score = critique.score
    return best_draft, False, cfg.max_iterations, best_score

Keep the critic stateless. Pass only the task, the current draft, and an explicit rubric. Stateful critics drift because they start referencing their own prior notes instead of the artifact.

3. Force structured critique output

Natural-language critiques are unparsable and unreliable for branching. Require JSON. A minimal schema:

{
  "approved": false,
  "score": 0.62,
  "issues": ["missing edge case for null input", "tone too casual"],
  "suggested_fix": "add guard clause and formalize language"
}

Validate before trusting the signal. In Python:

from pydantic import BaseModel, ValidationError

class Critique(BaseModel):
    approved: bool
    score: float
    issues: list[str]
    suggested_fix: str | None = None

def parse_critique(raw: str) -> Critique | None:
    try:
        return Critique.model_validate_json(raw)
    except ValidationError:
        return None

If validation fails, treat it as approved=False with a score of 0.0, not as a crash. A malformed critique that defaults to “keep going” can spin the loop to its hard ceiling every time.

4. Measure delta, not just absolute score

A score above threshold is sufficient to stop, but many tasks plateau below it. The agent reflection loop must detect when further revisions stop moving the needle. Track the last score; if |score - last_score| < min_delta for two consecutive steps, break. This prevents the classic failure where a model makes cosmetic edits that bump the score by 0.005 each turn.

Use a bounded scoring function. For text, a simple rubric-weighted sum works:

def score_draft(draft: str, rubric: dict) -> float:
    total = 0.0
    for criterion, weight in rubric.items():
        if criterion in draft:   # naive check; replace with real eval
            total += weight
    return min(total, 1.0)

For code, execute tests and map pass rate to score. Never let the generator score itself on the same axis it wrote.

5. Use a different model for the judge

Self-critique by the same weights is biased toward agreement. For objective tasks (code correctness, schema compliance), route the critic to a smaller, cheaper model or a different family. You trade nuance for independence.

When you split models, provider reliability becomes a concern. If you run the critic on a separate endpoint, an inference gateway that honors client routing directives—such as n4n.ai—lets you pin the judge to a specific model and automatically fall back when that provider is rate-limited, without rewriting your retry logic. The gateway also forwards cache-control hints, so repeated critique prompts with stable rubrics hit cache and cut cost.

Example client call with routing hint:

import openai

client = openai.OpenAI(base_url="https://api.n4n.ai/v1", api_key="KEY")
resp = client.chat.completions.create(
    model="anthropic/claude-3-haiku",
    messages=[{"role": "user", "content": critique_prompt}],
    extra_headers={"x-routing-pin": "judge-pool"}
)

The app code stays identical if the pinned pool degrades; the gateway shifts to a fallback.

6. Escalate instead of spinning

When the loop exits without convergence, return the best draft and a machine-readable status. Do not silently ship a half-revised answer.

result = {
    "draft": best_draft,
    "converged": bool,
    "iterations": i,
    "final_score": best_score,
    "issues": critique.issues if critique else []
}

Downstream code can decide to show a warning, request human review, or call a stronger model. The agent reflection loop is a local optimizer; it is not a guarantee of correctness.

7. Cap token spend explicitly

Reflection multiplies token usage by N iterations. Meter it. If you use per-token usage metering at the gateway layer, log usage.total_tokens per step and abort the loop when a budget is exceeded.

total_tokens = 0
for i in range(cfg.max_iterations):
    resp = generator.revise_with_usage(task, draft, critique)
    total_tokens += resp.usage.total_tokens
    if total_tokens > cfg.token_budget:
        break  # preserve remaining context for other calls

A token budget is often stricter than max_iterations because later revisions tend to have longer critiques and fuller drafts. Set the budget from your worst-case cost per call, not a guess.

8. Test the loop with simulated critics

Before relying on a live model, mock the critic to emit scripted scores: always-increasing, flat, oscillating, and crashing. Verify the loop stops at the right step and returns the best draft. This catches off-by-one errors in delta logic that only show up after you’ve paid for 10k tokens.

def fake_critic(scores):
    return lambda *a, **k: Critique(approved=False, score=next(scores), issues=[], suggested_fix=None)

Run it in CI. A reflection loop with untested stop logic is a latency bomb.

Common pitfalls

Critic and generator share context verbatim. The critic sees its own prior suggestions and rubber-stamps them. Strip prior critique text from the critic prompt; keep only the task and current draft.

Rubric too vague. “Make it better” yields random scores. Write the rubric as testable criteria: “Handles empty list, returns 400 on missing field, uses passive voice zero times.”

No validation on critique JSON. A malformed response flips the loop into an infinite revise cycle if you default to approved=False and never delta-check. Fail safe, not spin.

Ignoring latency in user-facing paths. A 5-step reflection loop adds 2–4 seconds worst case. For synchronous UX, drop max_iterations to 2 and use async background refinement.

Over-trusting score thresholds. A score of 0.81 on a broken SQL query is still broken. Use the critic score as a heuristic, not a validator. Run real tests where possible.

Forgetting to reset stale counter. If you only check delta against the immediate previous step but reset stale incorrectly, you may exit on a single flat step that follows a jump. Require two consecutive flat steps.

Tradeoffs

Quality vs cost. Each extra iteration buys marginal gains at linear token cost. Most improvement happens in iterations 1–2; beyond 4, you mostly pay for stagnation detection.

Independent judge vs same-model critic. A separate judge costs an extra model call but catches echo bias. For low-stakes text, same-model is fine; for code, separate is mandatory.

Strict delta stop vs hard max. Delta stopping saves tokens but can exit early on slow-ramp tasks. Keep the hard max as backstop.

Synchronous vs asynchronous reflection. Blocking loops simplify code but hurt latency. Fire the first draft immediately, then patch with a refined version if the loop converges within a budget.

Heuristic score vs executable validation. Executable checks (tests, schema validators) are stronger signals but require task-specific harnesses. Heuristics generalize but lie quietly.

An agent reflection loop is only as good as its stop logic. Build the termination contract first, measure deltas, separate the judge, and always return a status the caller can act on. Ship the loop with a budget, not a hope.