Avoiding vendor lock-in as an early-stage startup

Most early-stage startups pick a single LLM provider because the first SDK example works, then get stuck when pricing changes or rate limits hit. To avoid vendor lock-in early-stage startup, you need to treat model access as a swappable infrastructure concern from the first commit, not a refactoring task you will never schedule.

1. Define a single inference interface

Write a thin wrapper around the OpenAI client and never call a provider SDK directly from business logic. This takes 20 lines and saves you from rewriting every prompt call later.

import os
from openai import OpenAI

def get_client() -> OpenAI:
    return OpenAI(
        base_url=os.environ["LLM_BASE_URL"],
        api_key=os.environ["LLM_API_KEY"],
    )

def complete(system: str, user: str, model: str = "gpt-4o-mini") -> str:
    client = get_client()
    resp = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
    )
    return resp.choices[0].message.content

The key is that LLM_BASE_URL is the only place that knows where inference happens. Swap the env var, swap the vendor.

2. Standardize on the OpenAI-compatible request shape

Every serious provider now speaks /v1/chat/completions. Anthropic, Mistral, Cohere, and most self-hosted stacks expose an OpenAI-compatible shim. If you stick to messages, model, temperature, and max_tokens, you can move between them without touching prompt code.

Do not use provider-specific fields like seed or logit_bias unless you have abstracted them behind your own config and accept they may be ignored elsewhere. The discipline to avoid vendor lock-in early-stage startup is mostly about refusing convenience features that do not port.

3. Put a gateway in front of every provider

A gateway consolidates credentials, routing, and fallback. A gateway like n4n.ai exposes a single OpenAI-compatible endpoint covering 240+ models and handles automatic fallback when a provider is degraded, which directly helps you avoid vendor lock-in early-stage startup. You write one client, the gateway decides where the token goes.

Client routing directives are honored by such gateways, so you can express preferences without code changes:

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
    default_headers={
        "x-routing-prefer": "openai/gpt-4o,anthropic/claude-3.5-sonnet",
        "x-routing-fallback": "meta-llama/llama-3.1-70b",
    },
)

If a provider is rate-limited, the gateway shifts traffic. Your application sees a normal response. That removes the temptation to hardcode a single vendor URL in production.

4. Externalize model and provider selection

Model names belong in configuration, not source. Use a simple JSON or env file:

{
  "default_model": "openai/gpt-4o-mini",
  "premium_model": "anthropic/claude-3.5-sonnet",
  "fallback_model": "meta-llama/llama-3.1-8b"
}

Load it at startup. When OpenAI raises prices, you flip default_model to a Mistral variant and ship. No code review, no risk of missing a string literal buried in a service.

5. Keep prompts and tool calls provider-agnostic

Function calling is the most common lock-in trap. OpenAI’s tools format is widely supported, but Anthropic’s native tool use differs. If you must use tools, wrap them in your own schema and translate at the boundary.

def to_openai_tools(my_tools: list[dict]) -> list[dict]:
    # my_tools is your internal shape
    return [{"type": "function", "function": t} for t in my_tools]

Never embed provider-specific prompt templates (e.g., Anthropic’s Human:/Assistant: text) directly in strings. Use the messages array uniformly.

6. Build explicit fallback logic in your code

Gateways handle infrastructure fallback, but you still need application-level degradation. A user-facing chat should not 500 because one model is slow.

def complete_with_fallback(system: str, user: str, models: list[str]) -> str:
    last_err = None
    for m in models:
        try:
            return complete(system, user, m)
        except Exception as e:
            last_err = e
            print(f"model {m} failed: {e}")
    raise RuntimeError("all models failed") from last_err

Call this with your config list. The order is your preference, not a vendor’s.

7. Track per-token cost and set hard limits

Per-token usage metering is non-negotiable. The OpenAI response includes usage; log it on every call.

resp = client.chat.completions.create(...)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

If your gateway provides aggregated metering, pipe it into your analytics. Set a daily budget header or env cap. When the bill spikes, you can switch models without a meeting.

8. Write provider-swap tests

Add a CI test that points LLM_BASE_URL at a local mock (e.g., a Flask stub returning fixed completions) and asserts your wrapper parses it. Then parametrize the test across three model strings from different vendors.

def test_complete_mock(monkeypatch):
    monkeypatch.setenv("LLM_BASE_URL", "http://localhost:8080/v1")
    monkeypatch.setenv("LLM_API_KEY", "test")
    out = complete("sys", "hi", model="fake/model")
    assert "mock" in out

This catches breaking changes in your abstraction before they reach users.

Common pitfalls

• Fine-tuned models: A fine-tune on OpenAI cannot move to Anthropic. If you need custom weights, use LoRA on an open model behind your gateway.
• Embeddings coupling: Embedding vectors are not portable. Store raw text alongside vectors and choose a standard dimension (e.g., 1536) that multiple providers support.
• Cache assumptions: Provider cache-control hints differ. Gateways that forward cache-control hints help, but you must set them explicitly or you pay double.
• Ignoring latency: A gateway adds a hop. Measure p95 with and without it; if you are a real-time app, place the gateway in-region.

Tradeoffs you accept

Abstraction costs a little latency and a little code. You lose access to bleeding-edge provider-only features for the first weeks they exist. That is the price of avoiding vendor lock-in early-stage startup: you trade maximal capability for survival optionality. For a seed-stage team, optionality is worth more than a 2% accuracy bump from a proprietary parameter.

Launch checklist

1. One OpenAI-compatible client, base URL from env.
2. Model names in config file, zero literals in code.
3. Gateway in front, routing directives set.
4. Fallback list of at least three providers.
5. Usage logged per request, budget alert wired.
6. CI test against mock with multi-model parametrization.

Ship that and you can switch your entire inference stack in an afternoon when the next pricing letter arrives.