Evaluating tool-calling accuracy with promptfoo test cases

Tool-calling agents fail in ways that surface only in production: a model selects the right function but swaps two arguments, or silently falls back to text when a parameter is missing. Evaluating tool-calling accuracy promptfoo configurations forces those failures into a red/green test suite where you can pin them down. This guide walks through a concrete setup that asserts on function names, argument shapes, and semantic correctness.

Why tool-calling needs explicit assertions

Text similarity metrics like BLEU or cosine overlap are useless when the output is a JSON blob representing get_weather({location: "Berlin"}). You care about three things: did the model call a function at all, did it pick the correct function, and are the arguments valid against your schema? Promptfoo treats the model output as data, not prose, and lets you attach structured assertions.

A second reason: tool schemas evolve. When you add a unit parameter, older prompts may omit it. A test suite catches regressions immediately instead of after a user sees null in your API call.

Step 1: Install and scaffold promptfoo

Install the CLI globally or use npx. Then scaffold a config so you have a known file layout.

npm install -g promptfoo
promptfoo init

This creates promptfooconfig.yaml, prompts/, and tests/. You can delete the sample prompts; we will inline a minimal one. Verify the install with promptfoo --version (expects 0.70+ for stable function-call assertions).

Step 2: Configure the provider and tool schema

Define the model endpoint and the exact tool schema the model will see. When evaluating tool-calling accuracy promptfoo setups, keep the schema identical to what your agent sends at runtime—copy it from your code, don’t retype it.

If you route through an OpenRouter-class gateway like n4n.ai, set a single OpenAI-compatible base URL and let it handle fallback across 240+ models without changing your test file.

prompts:
  - "{{user_input}}"

providers:
  - id: openai:chat:anthropic/claude-3.5-sonnet
    config:
      apiBase: https://api.n4n.ai/v1
      apiKey: ${env:N4N_API_KEY}
      tools:
        - type: function
          function:
            name: get_weather
            parameters:
              type: object
              properties:
                location: { type: string }
                unit: { type: string, enum: [celsius, fahrenheit] }
              required: [location]

The id uses promptfoo’s openai:chat: prefix; the string after it is the model name your gateway expects. Api key comes from env to avoid committing secrets.

Step 3: Write baseline test cases

Create a test list that covers the happy path. Promptfoo supports is-valid-function-call (output parses and matches the supplied schema) and matches-function-call (name and supplied arguments equal the expected value).

tests:
  - description: "Weather query triggers get_weather"
    vars:
      user_input: "What is the weather in Tokyo in celsius?"
    assert:
      - type: is-valid-function-call
      - type: matches-function-call
        value:
          name: get_weather
          arguments:
            location: "Tokyo"
            unit: "celsius"

Multiple tools and routing

Add a second tool to the provider config and write a test that forces a choice:

      tools:
        - type: function
          function:
            name: get_weather
            ...
        - type: function
          function:
            name: set_alarm
            parameters:
              type: object
              properties:
                seconds: { type: integer }
              required: [seconds]

  - description: "Alarm query picks set_alarm not get_weather"
    vars:
      user_input: "Wake me in 30 minutes"
    assert:
      - type: matches-function-call
        value:
          name: set_alarm
          arguments:
            seconds: 1800

Negative tests

A robust suite also asserts when the model should not call a tool. Use reverse: true on a validity assertion:

  - description: "Casual chat emits no function call"
    vars:
      user_input: "Tell me a joke"
    assert:
      - type: is-valid-function-call
        reverse: true

Step 4: Assert on argument correctness with custom logic

Strict equality on arguments breaks when the model returns "Tokyo, JP" or "tokyo". For evaluating tool-calling accuracy promptfoo style at scale, drop to a Python assertion that applies your business rules.

  - description: "Tolerant location match"
    vars:
      user_input: "weather in tokyo jp, metric"
    assert:
      - type: python
        value: |
          import json
          def run(output, context):
              try:
                  call = json.loads(output)
              except Exception:
                  return False, "output is not json"
              if call.get("name") != "get_weather":
                  return False, "wrong function name"
              args = call.get("arguments", {})
              loc = args.get("location", "").lower()
              if "tokyo" not in loc:
                  return False, f"location mismatch: {loc}"
              if args.get("unit") not in ("celsius", "fahrenheit"):
                  return False, "bad unit"
              return True, "ok"

The run function must return a (bool, str) tuple. Promptfoo captures the string as the failure message.

Step 5: Run evaluations and read the report

Execute the suite and open the local viewer:

promptfoo eval --config promptfooconfig.yaml
promptfoo view

The table shows per-test pass/fail and the exact model output. If is-valid-function-call fails, the output pane shows the raw text—usually a markdown code fence or a natural-language refusal. Fix the provider tools block or adjust the prompt template, then re-run.

For CI you want machine-readable output:

promptfoo eval --output report.json --pass-rate-threshold 1.0

The --pass-rate-threshold flag exits non-zero if the overall pass rate drops below the given value.

Step 6: Wire into CI and track regressions

Add the eval to your GitHub Actions or GitLab CI after unit tests:

  - name: LLM tool-call eval
    run: |
      npm install -g promptfoo
      promptfoo eval --config promptfooconfig.yaml --output report.json --pass-rate-threshold 0.95
    env:
      N4N_API_KEY: ${{ secrets.N4N_API_KEY }}

Commit the promptfooconfig.yaml alongside the agent code. When a schema changes, update the config in the same PR; the diff shows exactly which test cases broke.

Scaling with var matrices

Promptfoo supports var combinations to generate many cases from a template:

tests:
  - description: "City coverage"
    vars:
      user_input: "weather in {{city}}"
    options:
      transform: "templates"
    # with a separate csv or yaml providing city list

This turns 5 hand-written tests into 50 city permutations without duplicating assertions.

Verifying success

Success means every test in promptfoo view is green and the CLI exits 0. Concretely:

• is-valid-function-call passes on all positive tool tests, proving the model emits schema-conformant JSON.
• matches-function-call confirms the correct function name and expected arguments for each intent.
• Negative tests with reverse: true stay red-free; the model does not hallucinate a call on chit-chat.
• Custom Python assertions return True, "ok" for tolerant matches.

If those hold and your --pass-rate-threshold is met, you have a reproducible gate that catches tool-calling drift before it reaches users.