A reliable GitHub Actions workflow for n4n apps does more than run unit tests—it validates that your LLM calls still succeed when a provider is degraded. This tutorial builds a pipeline that lints code, runs pytest, and executes a live smoke test against the OpenAI-compatible endpoint on n4n.ai, which addresses 240+ models and falls back automatically on provider errors.
Prerequisites
- A GitHub repository containing a Python LLM app that uses the OpenAI SDK pointed at a gateway.
- Python 3.11+ and pytest installed locally for testing.
- An API key for the gateway stored as
N4N_API_KEYin GitHub Actions secrets. - Familiarity with YAML and GitHub Actions syntax.
We’ll assume the app code lives in src/ and tests in tests/. The pattern works for any language that can call an OpenAI-compatible HTTP API, but the samples are Python.
Step 1: Structure the app
Create a minimal client that talks to the gateway. The OpenAI Python library accepts a base_url, so no custom HTTP code is needed.
# src/client.py
import os
from openai import OpenAI
def get_client(api_key: str) -> OpenAI:
return OpenAI(
api_key=api_key,
base_url="https://api.n4n.ai/v1", # OpenAI-compatible endpoint
default_headers={"X-App": "ci-smoke-test"},
)
def chat(prompt: str, model: str = "mistralai/mistral-7b-instruct") -> str:
client = get_client(os.environ["N4N_API_KEY"])
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
max_tokens=32,
)
return resp.choices[0].message.contentThe gateway forwards any provider cache-control hints you send via headers, so the default_headers are passed through untouched. Keep the default model small and cheap for CI.
Step 2: Write a smoke test
The smoke test should hit the live endpoint with a tiny model and assert we get tokens back. This catches credential, routing, and network regressions before they reach production.
# tests/test_smoke.py
import os
import pytest
from src.client import chat
def test_live_completion():
if not os.environ.get("N4N_API_KEY"):
pytest.skip("N4N_API_KEY not set")
out = chat("Say hello in one word.")
assert isinstance(out, str)
assert len(out.strip()) > 0Run it locally with pytest tests/test_smoke.py -s. Expected output:
============================= test session starts =============================
collected 1 item
tests/test_smoke.py . [100%]
============================== 1 passed in 0.42s ==============================If you omit the key locally, the test skips instead of failing. That same skip behavior will protect forks in CI.
Step 3: Define the GitHub Actions workflow
Create .github/workflows/ci.yml. The pipeline has three jobs: lint, test, and smoke. We separate smoke from unit tests because it requires secrets and network egress.
name: CI for n4n apps
on:
push:
branches: [main]
pull_request:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install ruff
- run: ruff check src tests
test:
runs-on: ubuntu-latest
needs: lint
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install pytest openai
- run: pytest tests/ -k "not live"
smoke:
runs-on: ubuntu-latest
needs: test
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
env:
N4N_API_KEY: ${{ secrets.N4N_API_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install pytest openai
- run: pytest tests/test_smoke.py -sThe if condition prevents running the live test on forks without the secret. This GitHub Actions workflow for n4n apps keeps unit tests fast and isolates external calls to a single guarded job. To speed up installs, add actions/cache for pip, but for a tiny dependency set it’s optional.
Step 4: Verify routing directives and cache hints
The gateway honors client routing directives and forwards provider cache-control hints. If your app sets Cache-Control in headers, the upstream provider receives it. You can extend the client to send those hints and confirm the call still succeeds.
# src/client.py (add)
def chat_with_cache(prompt: str, model: str = "mistralai/mistral-7b-instruct") -> str:
client = OpenAI(
api_key=os.environ["N4N_API_KEY"],
base_url="https://api.n4n.ai/v1",
default_headers={"Cache-Control": "max-age=3600"},
)
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
max_tokens=16,
)
return resp.choices[0].message.contentAdd a test that calls this and checks output. The gateway’s automatic fallback means even if the primary provider for that model is rate-limited, you still get a completion from a secondary provider—no code change required.
Step 5: Run the pipeline and read output
Push to main. GitHub Actions will show three sequential jobs. The smoke job log should contain:
============================= test session starts =============================
collected 1 item
tests/test_smoke.py . [100%]
============================== 1 passed in 0.51s ==============================If the key is missing, the test skips rather than fails. That’s intentional—PRs from forks shouldn’t break your CI. A more advanced GitHub Actions workflow for n4n apps might run the smoke test against multiple models in a matrix to detect per-model degradation:
strategy:
matrix:
model: ["mistralai/mistral-7b-instruct", "meta-llama/llama-3-8b-instruct"]Pass matrix.model into the test via an environment variable and parameterize the client call. This surfaces which model routes are healthy on every commit.
Step 6: Track per-token usage
The gateway provides per-token usage metering on every response. Capture it in the smoke test to surface cost signals in CI logs:
def test_usage_visible():
from src.client import get_client
client = get_client(os.environ["N4N_API_KEY"])
resp = client.chat.completions.create(
model="mistralai/mistral-7b-instruct",
messages=[{"role": "user", "content": "Hi"}],
max_tokens=8,
)
assert resp.usage.prompt_tokens > 0
assert resp.usage.completion_tokens > 0
print(f"prompt={resp.usage.prompt_tokens} completion={resp.usage.completion_tokens}")In Actions, the print appears under the pytest run, giving you a lightweight audit trail of token spend per commit. If you run the matrix from Step 5, you’ll see usage for each model separately.
Final notes
Keep the live smoke test small. A 32-token completion against a cheap model costs fractions of a cent, but a bloated prompt can add up across hundreds of PRs. The GitHub Actions workflow for n4n apps described here is deliberately minimal: lint, isolated unit tests, and one guarded live call. From here, add model matrices, response-schema validation, or latency guards as your app demands.