AI legal agents missing clauses in contracts rely on a pipeline that pairs a structured clause schema with a language model’s extraction and reasoning. This post walks through building that pipeline end to end, from defining required clauses to verifying the agent flags gaps accurately.
Step 1: Define the required clause schema
Start with a machine-readable contract playbook. List the clause types that must appear for a given document class, plus a natural-language spec for each. Store this as JSON so your agent can load it dynamically per contract type.
{
"contract_type": "mutual_nda",
"required_clauses": [
{
"id": "confidentiality",
"description": "Obligation to protect disclosed confidential information"
},
{
"id": "term",
"description": "Duration of the agreement and termination conditions"
},
{
"id": "indemnification",
"description": "Party bears costs for third-party claims arising from breach"
},
{
"id": "governing_law",
"description": "Specifies jurisdiction whose laws govern the contract"
}
]
}Engineers often skip this step and prompt the model with “find missing stuff.” That produces inconsistent output and makes regression testing impossible. A fixed schema gives you a deterministic diff surface. Version the schema in source control or a database table; legal teams will revise it as regulations change.
For enterprise contracts you may have conditional requirements: “if contract value > $1M, liability cap required.” Encode those as rules evaluated after extraction, not inside the LLM prompt. Keep the LLM focused on recognizing text, not applying business logic.
Step 2: Extract existing clauses from the contract
Run the contract text through an extraction call. Use JSON mode to force a structured list of found clauses with type, excerpt, and character span. Point the OpenAI client at any OpenAI-compatible endpoint.
from openai import OpenAI
import json
client = OpenAI(
base_url="https://api.n4n.ai/v1", # OpenAI-compatible, 240+ models, fallback built in
api_key="YOUR_KEY"
)
def extract_clauses(contract_text: str) -> list:
prompt = (
"Extract all legal clauses from the contract below. "
"For each, return id (snake_case type), excerpt, start_char, end_char. "
"If a clause does not match a known type, use 'other'."
)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": prompt},
{"role": "user", "content": contract_text}
],
response_format={"type": "json_object"}
)
data = json.loads(resp.choices[0].message.content)
return data["clauses"]The start_char/end_char fields let you cite the source later. Keep the raw text immutable; never mutate it before extraction. For contracts longer than the model context window, split on page breaks or section headers and extract per chunk, then merge by character offset. Track chunk boundaries to avoid double-counting a clause that spans a split.
Extraction quality depends on the model. Smaller models miss rare clause variants; larger ones cost more. Use the gateway’s per-token metering to compare cost per 100-page PDF and pick a default that meets your accuracy bar.
Step 3: Diff extracted clauses against the schema
A naive set difference fails because the model may label “Confidentiality” as conf instead of confidentiality. Use a second LLM call to map extracted clauses to schema ids semantically.
def map_to_schema(extracted: list, schema: dict) -> dict:
schema_ids = [c["id"] for c in schema["required_clauses"]]
prompt = (
f"Schema clause ids: {schema_ids}. "
"Given extracted clauses, return JSON mapping each extracted id to "
"the closest schema id, or null if none."
)
resp = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": prompt},
{"role": "user", "content": json.dumps(extracted)}
],
response_format={"type": "json_object"}
)
return json.loads(resp.choices[0].message.content)After mapping, compute missing ids:
mapped = map_to_schema(clauses, schema)
covered = {v for v in mapped.values() if v}
missing = [c["id"] for c in schema["required_clauses"] if c["id"] not in covered]This two-stage approach separates extraction from classification, which makes debugging tractable. If the agent misses a clause that is present, you inspect the extraction step alone. Add a confidence threshold: if the mapping response includes a score below 0.7, treat as uncovered and surface a “review manually” flag instead of a hard missing verdict.
Step 4: Generate flags with citations
For each missing id, produce a flag object that includes the schema description and a suggested remediation. You can template this without another model call.
def build_flags(missing: list, schema: dict) -> list:
desc_map = {c["id"]: c["description"] for c in schema["required_clauses"]}
flags = []
for mid in missing:
flags.append({
"missing_clause": mid,
"reason": f"Required clause not found: {desc_map[mid]}",
"severity": "high" if mid in ("indemnification", "governing_law") else "medium"
})
return flagsIf you need draft language, a final LLM call can generate a clause snippet given the contract context. Keep that call isolated so it doesn’t block flagging. Example output for a missing indemnification flag:
{
"missing_clause": "indemnification",
"reason": "Required clause not found: Party bears costs for third-party claims arising from breach",
"severity": "high"
}Attach the character spans of nearby sections so a human reviewer can jump straight to the gap. In a UI, render flags in the margin of the document using those offsets.
Step 5: Add resilience with a model gateway
Legal review jobs run in batches and hit rate limits. Pointing the client at an OpenAI-compatible gateway such as n4n.ai gives automatic fallback when a provider is degraded, and per-token usage metering keeps cost visible. The client code above already works; just set base_url and route directives via extra headers if you need a specific provider.
# honors client routing directives, forwards cache-control hints
client = OpenAI(
base_url="https://api.n4n.ai/v1",
api_key="YOUR_KEY",
default_headers={"x-n4n-route": "azure:gpt-4o-mini"}
)This is the only infrastructure change required to make the agent production-grade across 240+ models. When a primary provider returns 429, the gateway retries on a healthy route without code changes. For repeated contract templates, send cache-control: max-age=3600 to reuse prompt prefixes and cut latency.
Step 6: Verify the agent works
Write a test with a known-deficient contract. Use a short NDA missing indemnification and governing law.
def test_missing_clauses():
contract = """
MUTUAL NDA
The parties agree to keep confidential information secret.
This agreement lasts for two years.
"""
schema = json.load(open("mutual_nda.json"))
clauses = extract_clauses(contract)
mapped = map_to_schema(clauses, schema)
covered = {v for v in mapped.values() if v}
missing = [c["id"] for c in schema["required_clauses"] if c["id"] not in covered]
assert "indemnification" in missing
assert "governing_law" in missing
flags = build_flags(missing, schema)
assert len(flags) == 2Run pytest. If the assertions pass, your AI legal agents missing clauses detection fires correctly. For CI, mock the LLM client with recorded responses to keep tests fast and free. Capture a golden contract set with known gaps and assert flag counts stay stable across model upgrades.
Integration verification extends beyond unit tests. Deploy the agent to a staging bucket of 50 real NDAs with manually labeled missing clauses. Measure precision and recall; tune the mapping prompt if recall drops below 0.95. Log every token via the gateway’s metering to spot prompt bloat.
Production considerations
Cache extraction results per contract hash. Clause schemas evolve; version them in a database. Log every token via the gateway’s metering to spot prompt bloat. With these steps, you have a deterministic, auditable missing-clause detector rather than a black box.
When you process amendments, run extraction on the redline diff only and merge with the base contract’s clause map. That avoids re-flagging clauses that were present before and removed intentionally. Treat removal as a separate event type so the agent reports “clause deleted” distinct from “clause missing.”
AI legal agents missing clauses in contracts become trustworthy only when the schema, extraction, and verification are separated and tested. Ship the pipeline above, then iterate on model choice and prompt wording using real review data.