Deploy a LiteLLM Proxy to Unify Every LLM Provider (2026)

Every team that uses more than one LLM provider eventually builds the same thing badly: a mess of if-statements switching between OpenAI, Anthropic, and some open-weight model, each with its own SDK, its own auth, its own retry quirks. LiteLLM (https://docs.litellm.ai) is the tool that deletes that mess. Its proxy server gives you one OpenAI-compatible endpoint in front of 100+ providers, plus virtual API keys, per-key rate limits, spend tracking, and fallbacks. I run PandaStack; here's how to self-host the LiteLLM proxy properly, with the Postgres and Redis it wants.

What the LiteLLM proxy does for you

• One endpoint, many providers. Your app calls /v1/chat/completions; LiteLLM routes to OpenAI, Anthropic, Azure, Bedrock, your self-hosted vLLM, whatever — based on the model name.
• Virtual keys. Issue per-team or per-app keys with their own budgets and rate limits, so one runaway script can't burn your whole month.
• Fallbacks and load balancing. If a provider errors or rate-limits, LiteLLM retries against a fallback model automatically.
• Cost tracking. Spend logged per key/model, backed by a database.

That last point is why we add Postgres: durable spend and key data. Redis makes rate limiting fast and consistent across replicas.

Step 1: The config

LiteLLM's proxy is driven by a config.yaml mapping your model names to providers:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  - model_name: claude
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: local-mistral
    litellm_params:
      model: openai/mistralai/Mistral-7B-Instruct-v0.3
      api_base: https://your-vllm.pandastack.io/v1
      api_key: os.environ/VLLM_KEY

litellm_settings:
  drop_params: true

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Note local-mistral pointing at a self-hosted vLLM server — you can mix hosted providers and your own inference behind the same proxy. Provider keys come from environment variables, never the file.

Step 2: Dockerfile

LiteLLM ships an official image:

FROM ghcr.io/berriai/litellm:main-latest
COPY config.yaml /app/config.yaml
EXPOSE 4000
CMD ["--config", "/app/config.yaml", "--port", "4000", "--host", "0.0.0.0"]

Step 3: Provision Postgres and Redis

On PandaStack:

1. 1Databases → New Database → PostgreSQL — for keys, budgets, and spend logs. Attach it to the app so DATABASE_URL is injected.
2. 2Databases → New Database → Redis — for fast, shared rate limiting across replicas. Set its URL as an env var (e.g. REDIS_URL).

Both are managed — backups, SSL, monitoring included — so you're not babysitting a Redis box.

Step 4: Deploy on PandaStack

1. 1Push the repo (Dockerfile + config.yaml) to Git.
2. 2https://dashboard.pandastack.io → New App → connect repo. It builds and deploys.
3. 3Set encrypted environment variables:

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
LITELLM_MASTER_KEY=sk-your-own-master-key
REDIS_URL=redis://...        # from the managed Redis
# DATABASE_URL is injected by the attached Postgres

1. 1Expose port 4000; add a custom domain with automatic SSL.

CLI:

npm install -g @pandastack/cli
panda login
panda deploy

Step 5: Use it from anywhere

Your entire codebase now targets one endpoint:

from openai import OpenAI
client = OpenAI(base_url="https://your-litellm.pandastack.io", api_key="sk-virtual-key")

# switch providers by changing ONE string
client.chat.completions.create(model="gpt-4o", messages=[...])
client.chat.completions.create(model="claude", messages=[...])
client.chat.completions.create(model="local-mistral", messages=[...])

Same SDK, same request, different model name. Provider migration becomes a config edit, not a code rewrite.

Step 6: Issue virtual keys

With the master key, mint scoped keys for teams:

curl https://your-litellm.pandastack.io/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"models": ["gpt-4o", "claude"], "max_budget": 50, "duration": "30d"}'

Now the mobile team gets a key capped at $50/month that can't touch your local model. Spend shows up in the database, viewable in LiteLLM's UI.

Honest tradeoffs

• It's another hop. The proxy adds a small amount of latency versus calling a provider directly. In exchange you get routing, budgets, fallbacks, and one integration surface — almost always worth it past two providers.
• You now run infrastructure (proxy + Postgres + Redis). If you use exactly one provider and don't need key management, the raw provider SDK is simpler.
• Pin the image tag; LiteLLM moves quickly.

Wrap-up

LiteLLM's proxy turns provider chaos into one OpenAI-compatible endpoint with keys, budgets, and cost tracking. Run it on PandaStack with managed Postgres and Redis, mix hosted and self-hosted models behind it, and never rewrite provider glue again.

Docs: https://docs.pandastack.io. Start free: https://dashboard.pandastack.io.