Gemini 3's native tool use for agentic workflows

Gemini 3 tool use changes how you build agentic loops: the model emits structured function calls natively, without prompt hacking or fragile string parsing. This guide gives an ordered path to wire those calls into a production agent that queries APIs, reasons over results, and recovers from failures.

1. Define tools as strict schemas

Start by writing function declarations with tight JSON Schema. Loose schemas invite malformed arguments. Gemini 3 tool use expects exact schema matches, so required parameters must be non-nullable and described precisely.

{
  "name": "get_weather",
  "description": "Fetch current weather for a lat/lng point",
  "parameters": {
    "type": "object",
    "properties": {
      "lat": {"type": "number", "description": "Latitude in degrees"},
      "lng": {"type": "number", "description": "Longitude in degrees"}
    },
    "required": ["lat", "lng"]
  }
}

Register the tool in the tools field of the request. Avoid varargs or free-form additionalProperties: true unless you genuinely need them. The model will fill what you specify; ambiguity costs you validation code later. In practice, agents break because a tool accepted city as a string but the model passed coordinates. Pin types and constrain enums where possible.

2. Bootstrap the conversation with a system instruction

Gemini separates system instructions from user turns. Use it to set agentic constraints: which tools are safe, when to stop, and how to format final answers. Putting constraints in systemInstruction keeps them out of the mutable user turn, reducing prompt injection surface.

import requests, json

URL = "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.0-pro:generateContent"
HEADERS = {"Content-Type": "application/json"}
# api_key injected from env in real code
payload = {
  "systemInstruction": {
    "parts": [{"text": "You are an ops agent. Use get_weather to answer location questions. Never invent data. If a tool fails, say so."}]
  },
  "tools": [{"function_declarations": [{
    "name": "get_weather",
    "description": "Fetch current weather for a lat/lng point",
    "parameters": {
      "type": "object",
      "properties": {
        "lat": {"type": "number"},
        "lng": {"type": "number"}
      },
      "required": ["lat", "lng"]
    }
  }]}],
  "contents": [{
    "role": "user",
    "parts": [{"text": "What's the weather at 37.77, -122.41?"}]
  }]
}
r = requests.post(f"{URL}?key=YOUR_KEY", headers=HEADERS, data=json.dumps(payload))
print(r.json()["candidates"][0]["content"]["parts"])

The response contains a functionCall part when the model wants to invoke a tool. No separate “agent mode” flag exists; the native tool use is just part of the generation.

3. Execute the tool call loop

Parse the functionCall, dispatch to your local implementation, and return a functionResponse as a new turn. The model continues reasoning with that data in context. This loop is the core of Gemini 3 tool use.

def get_weather(lat, lng):
    # real implementation calls a weather API
    return {"temp_c": 14, "condition": "fog"}

resp = r.json()
part = resp["candidates"][0]["content"]["parts"][0]
if "functionCall" in part:
    fn = part["functionCall"]
    args = fn["args"]
    result = get_weather(args["lat"], args["lng"])
    followup = {
        "contents": [
            resp["candidates"][0]["content"],  # model's turn with the call
            {
                "role": "user",
                "parts": [{
                    "functionResponse": {
                        "name": fn["name"],
                        "response": {"result": result}
                    }
                }]
            }
        ]
    }
    r2 = requests.post(f"{URL}?key=YOUR_KEY", headers=HEADERS, data=json.dumps(followup))
    print(r2.json()["candidates"][0]["content"]["parts"])

Keep the loop bounded. Set a max iteration count (e.g., 5) to prevent runaway chains. Gemini 3 tool use is reliable but not infallible; a confused model can repeat the same call. If the model returns text instead of a functionCall, treat that as the final answer and exit.

A key advantage of Gemini 3 native tool use is that functionResponse can carry images or audio via inline_data. If your tool renders a chart, return it directly instead of a text description.

{
  "role": "user",
  "parts": [{
    "functionResponse": {
      "name": "render_chart",
      "response": {
        "image": {
          "inline_data": {
            "mime_type": "image/png",
            "data": "base64encodedstring"
          }
        }
      }
    }
  }]
}

Tradeoff: multi-modal blobs inflate context token counts. Strip them after the model acknowledges receipt if you don’t need them in later steps. Don’t let every intermediate frame accumulate. A 5-step agent with image returns can blow past context limits fast.

5. Implement fallback and retries

Two failure modes dominate: provider 503s and model emitting an unknown tool name. For the first, route through a gateway that retries across providers. If you route through n4n.ai, you get one OpenAI-compatible endpoint that addresses 240+ models and automatic fallback when a provider is rate-limited, so a Gemini outage doesn’t kill your agent.

For the second, validate functionCall.name against your registry before dispatch:

ALLOWED = {"get_weather", "render_chart"}
if fn["name"] not in ALLOWED:
    # return error as functionResponse, let model recover
    result = {"error": "unknown tool"}

Never execute side-effecting tools (DB writes, emails) without a confirmation gate. Despite robust Gemini 3 tool use, the model will happily call send_email if you declare it; that’s on you. Set timeouts on your local tool HTTP clients; a hung downstream API stalls the whole loop.

6. Stream intermediate steps to users

Agents feel opaque if the UI freezes during tool calls. Surface functionCall names as “Calling get_weather…” and show truncated responses. The GenerateContent API supports streamGenerateContent for token streaming, but tool calls arrive as discrete parts. Render them as timeline events.

curl -N "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.0-pro:streamGenerateContent?key=YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d @payload.json

Consume the stream, buffer parts, and emit a WebSocket event per functionCall or text segment. Users trust agents more when they can see the work.

7. Common pitfalls and tradeoffs

Latency: Each tool round-trip adds a full model inference. Parallelize independent calls only if you fork contexts—Gemini’s native loop is sequential by default.

Cost: Function schemas and responses live in context for every subsequent step. Keep tool outputs lean; return IDs not full objects. A multi-step agent can balloon context to tens of kilobytes per run.

Over-delegation: Developers often expose 20 tools at once. The model’s selection accuracy drops. Group tools by domain and swap the tools array based on conversation state.

Validation debt: Native tool use removes prompt parsing, not validation. You still must check argument types, ranges, and authz. Treat the model as a junior dev who writes plausible but untrusted code.

State management: The contents array is your agent’s memory. Truncate old tool responses or summarize them; otherwise you hit context limits within a dozen steps.

Gemini 3 tool use gives you a clean primitive. The engineering work is everything around it: schemas, loops, guardrails, and UX. Ship the boring parts first.