Open-Source & Self-Hosted LLM Gateways

5 Platforms

1. LiteLLM

   Open-source AI gateway and LLM proxy putting 140+ providers and 1,800+ models behind one OpenAI-compatible key, with spend tracking, smart routing, and self-hosting anywhere, even air-gapped.

   Visit →

2. Bifrost

   Open-source (Apache 2.0), self-hostable AI gateway unifying 1,000+ models across 8+ providers behind a drop-in OpenAI/Anthropic-compatible SDK — with provider failover, budgeting, MCP gateway support, and built-in observability.

   Visit →

3. Otari (Mozilla.ai)

   Open-source, OpenAI-compatible LLM gateway you run yourself, built on Mozilla.ai’s any-llm, with virtual keys, budgets and usage tracking across 40+ providers; a hosted option adds routing, observability and team management.

   Visit →

4. LLM Gateway

   Open-source (AGPLv3), self-hostable unified API for 40+ LLM providers with real-time cost analytics, automatic failover, and drop-in OpenAI SDK compatibility; bring-your-own-keys tier is free.

   Visit →

5. MLflow AI Gateway

   Gateway built into the open-source MLflow AI platform, exposing an OpenAI-compatible endpoint for any LLM provider with native tracing, evaluation, and MCP server access management, licensed Apache 2.0.

   Visit →

Managed & Hosted LLM Gateways

5 Platforms

1. OpenRouter

   Unified interface for 400+ models across 70+ providers, letting developers find the best model and price for any prompt with automatic fallback and edge-optimized routing.

   Visit →

2. Cloudflare AI Gateway

   Intelligent control plane for AI applications offering dynamic model routing, response caching, usage analytics, and unified billing across providers, built on Cloudflare's global network.

   Visit →

3. Vercel AI Gateway

   One endpoint for hundreds of AI models from Vercel, without managing separate rate limits and API keys for each provider.

   Visit →

4. Requesty

   OpenAI-compatible router for 600+ models, with smart routing, caching, failover, observability and EU data residency.

   Visit →

5. Portkey

   LLM gateway with routing, guardrails, observability and governance, available as open-source and hosted. Palo Alto Networks completed its acquisition in May 2026.

   Visit →

Secure & Compliance-Focused LLM Gateways

1 Platform

1. SUPERWISE Sentinel

   Commercial LLM gateway that redacts PII and secrets from every request inside your own network before the prompt reaches a public LLM, for teams with data-protection and compliance requirements.

   Visit →

LLM Routing & Cost-Optimization Frameworks

2 Platforms

1. RouteLLM

   Open-source framework for serving and evaluating LLM routers, which send easy queries to cheaper models to cut costs without losing quality.

   Visit →

2. vLLM Semantic Router

   Programmable mixture-of-models router from the vLLM project that picks the right model for each request across heterogeneous LLM inference.

   Visit →