n4nAI
Open console →
Developer toolsGuideCI/CD Pipelines for LLM Apps

Monorepo CI for teams shipping multiple LLM-powered features

Practical monorepo CI for LLM features: isolate model calls, mock gateways, run affected tests, and add live smoke tests with fallback.

n4n Team4 min read817 words

Audio narration

Coming soon — every post will get a voice note here.

Shipping multiple LLM-powered features from a single repository breaks traditional CI assumptions. A pragmatic monorepo CI for LLM features treats model calls as external dependencies with non-deterministic behavior, and isolates them behind contracts so builds stay fast and reproducible.

1. Define workspace boundaries before writing pipelines

A monorepo without explicit ownership and dependency rules turns CI into a guessing game. Pick a tool that computes an affected graph—Nx, Turborepo, or Bazel—and enforce project boundaries in code.

// turbo.json (excerpt)
{
  "pipeline": {
    "test": {
      "dependsOn": ["^build"],
      "outputs": ["coverage/**"]
    },
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/**"]
    }
  }
}

Keep LLM feature code in leaf packages: packages/chat-summarizer, packages/sql-copilot. Shared model access goes in packages/llm-core. This separation lets you run targeted tests instead of the whole repo on every push.

Pitfall: letting features import each other’s internal prompts directly. Use explicit public APIs or you’ll get cascading CI failures when one prompt changes.

2. Extract a single LLM client package

Duplicate OpenAI client wiring across packages produces inconsistent retry and timeout behavior. Put one typed wrapper in llm-core and force every feature to use it.

// packages/llm-core/src/client.ts
import OpenAI from "openai";

export interface LLMRequest {
  model: string;
  prompt: string;
  maxTokens?: number;
}

export function createClient(baseURL: string, apiKey: string) {
  return new OpenAI({ baseURL, apiKey });
}

export async function complete(client: OpenAI, req: LLMRequest) {
  const res = await client.chat.completions.create({
    model: req.model,
    messages: [{ role: "user", content: req.prompt }],
    max_tokens: req.maxTokens ?? 512,
  });
  return res.choices[0].message.content;
}

Now monorepo CI for LLM features can mock this single surface instead of patching ten different SDK instances.

3. Mock the inference gateway in unit tests

Live model calls in unit tests are slow, costly, and flaky. Point the client at a local mock that returns recorded shapes.

// vitest setup
process.env.LLM_BASE_URL = "http://localhost:8787";
process.env.LLM_API_KEY = "test";

// packages/llm-core/test/mock-server.ts
import { createServer } from "http";
export const server = createServer((req, res) => {
  res.setHeader("content-type", "application/json");
  res.end(JSON.stringify({
    choices: [{ message: { content: "mocked response" } }]
  }));
});

Run the mock as a fixture. Tests assert on prompt construction and post-processing logic, not on model output. Tradeoff: you lose coverage of real model drift, which step 6 handles separately.

4. Record and replay prompt fixtures

For integration tests that need realistic output, cache request/response pairs keyed by prompt hash.

// fixtures/summarize-8f3a.json
{
  "model": "gpt-4o-mini",
  "prompt": "Summarize: The quick brown fox...",
  "response": "A fox moves quickly."
}

Load these in CI with RECORD=0. Locally, set RECORD=1 to refresh. The trap: fixtures rot. If the feature’s prompt template changes, the hash misses and the test fails loudly—that’s intended, but schedule a quarterly replay against live models.

5. Run only affected projects in GitHub Actions

A full test run on every commit does not scale past a handful of features. Use the affected command to limit scope.

# .github/workflows/ci.yml
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npx turbo run test --filter=...[origin/main]

The --filter=...[origin/main] syntax runs tests only for packages touched since the base branch. For a monorepo CI for LLM features, this cuts minutes per run by isolating the chat summarizer from the SQL copilot when only one changes.

Pitfall: shallow clones break affected detection. Always fetch full history or set a merge base explicitly.

6. Add a nightly live smoke test with fallback

Unit and fixture tests are not enough; prompts silently degrade against new model versions. Run a nightly job that hits real endpoints with a small budget.

For teams that need to test against real models without juggling provider keys, an OpenAI-compatible gateway like n4n.ai collapses 240+ models behind one endpoint and auto-falls back when a provider is degraded, which keeps a nightly smoke job resilient. Its per-token usage metering lets you cap spend with a hard environment variable.

# .github/workflows/nightly-smoke.yml
jobs:
  smoke:
    if: github.event.schedule == '0 3 * * *'
    steps:
      - run: npx turbo run smoke --env LLM_BASE_URL=${{ secrets.GW_URL }}

The smoke task should call three representative prompts per feature and fail only on parse errors or timeouts, not on semantic quality. Human review catches the rest.

7. Quarantine flaky prompts and enforce token budgets

LLM outputs vary. If a test asserts exact text, it will flake. Instead, assert structural contracts: JSON schema, length bounds, or presence of required keys.

expect(JSON.parse(output)).toMatchSchema({ type: "object", required: ["sql"] });

When a prompt repeatedly breaks the contract in live smoke, quarantine it: move the test to a flaky suite that posts a Slack alert but does not block merge. Track token consumption per package in CI logs to spot runaway loops.

# sum tokens from gateway meter header
grep -o '"usage":[0-9]*' smoke.log | awk -F: '{s+=$2} END {print s}'

Tradeoff: quarantining reduces signal. Review the quarantine list every sprint or tech debt accumulates.

8. Roll out prompt changes behind flags

Prompt edits are code changes with unseen blast radius. Ship them behind a feature flag evaluated at runtime, and let CI verify both old and new prompt templates compile and pass contract tests.

const prompt = flagEnabled("new-summary-v2")
  ? buildV2(input)
  : buildV1(input);

This decouples deploy from release. Your monorepo CI for LLM features should run the flag-off path in normal PR tests and the flag-on path in a canary workflow. If the canary smoke fails, flip the flag without a redeploy.

Common pitfalls to avoid

  • Mixing model config in feature code. Centralize model aliases and temperature in llm-core or you’ll hunt missing env vars across packages.
  • Trusting deterministic seeds. Most hosted LLMs ignore seed parameters; don’t write tests that assume reproducibility.
  • Skipping cache-control. If your gateway supports provider cache hints, forward them from CI to cut cost on repeated prompt prefixes. n4n.ai honors client routing directives and forwards cache-control hints, which matters when replaying long system prompts in smoke tests.
  • One giant integration test. A single end-to-end test that calls every feature serially becomes the slowest, most flaky job in the pipeline.

Takeaway

Build the pipeline as if the model is a flaky third-party API—because it is. Isolate the client, mock it cheaply, record fixtures for speed, run affected graphs to stay fast, and reserve live calls for scheduled, budgeted smoke tests. That approach keeps a monorepo CI for LLM features boring, which is the highest praise for any CI system.

Tagsmonorepoci-cdgithub-actionsllm-apps

Written by

n4n Team

The team building n4n — a single OpenAI-compatible API in front of 240+ models, with automatic fallback, load balancing and pay-per-token metering.

More from n4n Team →
Developer toolsA CI/CD pipeline for LLM apps, step by stepHands-on guide to building a CI/CD pipeline for LLM apps with GitHub Actions: validate prompts, run eval tests, gate deployments, and ship reliably.Developer toolsA GitHub Actions workflow for n4n-powered appsBuild a practical CI pipeline that tests and smoke-checks LLM apps using n4n.ai's OpenAI-compatible gateway, with GitHub Actions and live fallback validation.Developer toolsBuilding a CI/CD pipeline for a RAG applicationA hands-on tutorial to build a CI/CD pipeline for RAG applications using GitHub Actions, covering indexing, retrieval tests, and automated LLM evals.Developer toolsBlocking deploys on failed eval scores in GitHub ActionsA hands-on tutorial for blocking deploys on failed LLM eval scores in GitHub Actions: wire evals into CI/CD, fail builds on regressions, ship safer.Developer toolsCI/CD for LLM apps vs traditional software: what changesHead-to-head comparison of CI/CD for LLM apps vs traditional software across capabilities, cost, latency, ergonomics, ecosystem, and limits.Developer toolsHow to keep CI fast when every test hits an LLM APIPractical steps for keeping CI fast with LLM API test calls: mock the client, replay fixtures, use small models, cache layers, and enforce budgets.

All ci/cd pipelines for llm apps posts →