LiteLLM Alternatives: 8 LLM Gateways Compared (2026)

The best LiteLLM alternatives depend on why you're looking. For a self-hosted, open-source gateway that isn't a Python package, pick Bifrost. For guardrails in the gateway, pick Portkey. If your platform team already runs Kubernetes and Envoy, pick Agent Router (formerly Envoy AI Gateway). If you'd rather not run a proxy at all, use Cloudflare AI Gateway or Vercel AI Gateway. If you already run Kong for your APIs, use Kong AI Gateway.

Most teams searching for a LiteLLM alternative still like what LiteLLM does: one OpenAI-compatible endpoint in front of every model provider, with keys and budgets in one place. What changed is how much they trust the layer that holds every provider key. This guide covers what happened at LiteLLM in 2026, then compares eight gateways on licence, hosting and ownership.

Quick Comparison

Why Teams Look for a LiteLLM Alternative

LiteLLM is healthy. Its GitHub repo had commits on the day we checked, and it calls itself an open-source AI gateway for 100+ LLMs. Three things push teams to look around anyway.

1. The March 2026 supply-chain incident. This is the reason most people have heard of, so here is exactly what LiteLLM's own security update says:

• The compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8.
• They were live on March 24, 2026 from 10:39 UTC "for about 40 minutes before being quarantined by PyPI."
• An attacker "bypassed official CI/CD workflows and uploaded malicious packages directly to PyPI." LiteLLM says it believes the compromise began with the Trivy dependency in its CI/CD security scanning workflow.
• The GitHub advisory, rated Critical, describes the payload as credential-harvesting malware that sent data to a remote API.
• Not affected: the official LiteLLM proxy Docker image, LiteLLM Cloud, installs from source on GitHub, and anyone on 1.82.6 or earlier who didn't upgrade in that window.
• Affected: anyone who ran an unpinned pip install litellm during the window, including Docker builds and projects that pulled LiteLLM in as an unpinned dependency.

LiteLLM's response is also on that page. It rotated maintainer credentials, brought in Google's Mandiant for forensics, paused releases, and shipped a clean v1.83.0 on March 30 through a rebuilt release pipeline. Its Docker images on GHCR are now signed with cosign. The page was still labelled an active investigation when we read it in November 2026.

The fair reading: this was a publishing-pipeline compromise, not a flaw in the gateway code, and LiteLLM says no malicious code reached its main branch. But a gateway holds every provider key you own, so security teams now ask harder questions about how it's shipped.

2. Enterprise features need a licence key. LiteLLM's LICENSE file is MIT for everything outside the enterprise/ directory. Its enterprise docs list what sits behind a key: SCIM, audit logs, role-based access control, IP allowlists, secret-manager integrations and several guardrail integrations. SSO is free only up to five users. If those are must-haves, you're comparing commercial products either way.

3. You're running production infrastructure. The proxy needs a database for spend tracking, and Redis if you want shared caching. Upgrades and uptime are yours.

Where LiteLLM Is Still the Right Call

• Breadth. Provider coverage is LiteLLM's strongest point, and many agent frameworks ship a LiteLLM adapter.
• Two forms. You can use the Python SDK inside your code, the proxy as a central service, or both.
• Proxy features in the MIT core. Virtual keys, per-project spend tracking, load balancing, guardrails and an admin dashboard are listed in the README, along with an MCP gateway.

If the incident is your only worry, pinning versions, verifying the published checksums and running the signed Docker image may be less work than migrating.

Open-Source LiteLLM Alternatives You Self-Host

These three are the answer if you searched for an open source LLM gateway or a LiteLLM proxy alternative you can run inside your own network.

Bifrost: a gateway that isn't a Python package

Best for: teams that want LiteLLM's proxy features in a standalone service.

Bifrost is an Apache-2.0 gateway from Maxim. Its README says it unifies 23+ providers behind one OpenAI-compatible API, and you start it with one npx command or a Docker image, then configure it in a web UI. Listed features map closely to LiteLLM's proxy:

• Automatic fallbacks between providers and models
• Load balancing across API keys and providers
• Semantic caching based on similarity, not only exact matches
• Budgets through virtual keys, teams and customers
• Prometheus metrics and distributed tracing

It also has a Go SDK if you'd rather embed it. Maxim publishes microsecond-level overhead benchmarks in the README. Those are vendor numbers, so test on your own traffic.

The trade-off: the README's enterprise section lists adaptive load balancing, clustering, guardrails and an MCP gateway as private-deployment features. Check which side of that line your requirements fall on. Provider coverage is also narrower than LiteLLM's.

Portkey: guardrails first, now under Palo Alto Networks

Best for: teams whose main concern is checking what goes into and out of models.

The Portkey gateway is MIT-licensed and describes itself as an AI gateway with integrated guardrails. It handles fallbacks, automatic retries, weighted load balancing and response caching, and you can run it locally with npx, in Docker or on Cloudflare Workers.

Ownership changed this year. Palo Alto Networks closed its acquisition of Portkey on May 29, 2026, calling it the core AI gateway for Prisma AIRS. Portkey's own site now carries a banner saying Portkey is now Prisma AIRS AI Gateway.

The trade-off: the repo is still MIT and not archived, but its main branch showed no commits after May 25, 2026 when we checked on November 12. The README says Portkey's enterprise gateway is merging into open source in a 2.0 release, which lives on a pre-release branch. Before you commit to the open-source version, look at where active development is happening.

Agent Router: for platform teams on Kubernetes

Best for: companies that already run Envoy and want AI traffic managed the same way as everything else.

You may know this project by its old name. Agent Router was Envoy AI Gateway until September 2026, when it joined the Agentic AI Foundation under a new name. The project says it's the same code, the same maintainers and the same Apache-2.0 licence, and that old repository links redirect.

It's a control plane on top of Envoy Gateway. You declare providers, credentials, token limits and failover as Kubernetes resources, and Envoy carries the traffic. The site lists 22 providers, an MCP gateway that merges many MCP servers into one tool catalog, and metrics that follow OpenTelemetry's GenAI conventions.

One detail matters for LiteLLM users in particular. On November 6, 2026, the project announced a Python SDK that "keeps LiteLLM's API and sends every call through Agent Router," moving fallbacks, retries and provider keys out of your Python process.

The trade-off: this is Kubernetes-native infrastructure. If you don't have a platform team, it's more machinery than a single proxy container.

Managed and Enterprise Gateways

Kong AI Gateway

Best for: organisations that already run Kong for their APIs.

Kong AI Gateway governs LLM traffic, MCP tool exposure and agent-to-agent traffic from the same place as your other APIs. Its docs list routing, load balancing and failover across providers, a semantic cache, prompt guards, PII sanitising and AI rate limiting tied to spend. You can run it through Kong Konnect or self-hosted.

The trade-off: it's a large platform, and the docs note that AI Gateway 2.x replaces the older plugin-centric setup with a new entity model. Confirm with Kong which licence your chosen features need.

Cloudflare AI Gateway

Best for: adding logs, caching and rate limits in front of provider accounts you already have.

Cloudflare AI Gateway is available on all Cloudflare plans. You get analytics on requests, tokens and cost, logging, caching, rate limiting, retries and model fallback. Its pricing page says core features are offered for free; log storage beyond your plan's limits and exports are billed separately, and paying for tokens through Cloudflare adds a fee when you buy credit.

The trade-off: caching is exact-match only, and prompts pass through Cloudflare's network. That's the opposite of self-hosting.

Vercel AI Gateway

Best for: teams that want budgets and failover without operating anything.

Vercel AI Gateway is a hosted API you can call from any server or cloud. Your app doesn't need to run on Vercel. It accepts the OpenAI Chat Completions, OpenAI Responses and Anthropic Messages formats, logs every routing attempt, and lets you set budgets per team, project, API key or member. Vercel's docs say it adds zero markup to provider token prices, including when you bring your own keys.

The trade-off: budgets only cover spend billed through Vercel's credentials, and its caching is provider prompt caching, not full-response caching.

Helicone AI Gateway and OpenRouter

Two more names show up on most lists, each with a caveat.

Helicone has an Apache-2.0 AI gateway that pairs routing with request logging. But Mintlify announced it was acquiring Helicone in March 2026 and said the product would run in maintenance mode, and Helicone's site now says it is no longer part of Mintlify. We cover that, and what to use instead, in Helicone alternatives.

OpenRouter is a hosted model marketplace: one key, one bill, hundreds of models on OpenRouter's provider accounts. That's a different job from a gateway in front of your own keys. If that's the shape you want, our OpenRouter alternatives guide compares it with its rivals.

A note on TensorZero: older lists recommend it as a Rust-based gateway. Its GitHub repository was archived by its owner on June 12, 2026 and is now read-only.

How to Choose a LiteLLM Alternative

Whichever gateway you pick, the March incident leaves a checklist that applies to all of them:

1. Pin versions. Unpinned installs were the ones affected.
2. Prefer signed artefacts. Use official container images and verify signatures or checksums where the vendor publishes them.
3. Scope the keys. Give the gateway provider keys with spend limits, and rotate them on a schedule.
4. Check who owns it today. Three gateways on this page changed owner or name in 2026.
5. Keep the OpenAI format. Most gateways here accept it, so switching later is a base-URL change, not a rewrite.

A gateway's logs are a start, not a full tracing tool. If you need evals and prompt management on top, see our Langfuse alternatives roundup.

Where Taku Fits

Taku isn't an LLM gateway, so it doesn't replace LiteLLM or anything else on this list. Gateways decide which model answers a request. They don't help with the earlier step: finding an AI setup worth running, and getting it working without building the stack yourself.

That's where Taku comes in. Taku Apps and Sites, built by the best AI minds. Find one, run it, and keep it in your Mac Dock. Taku is in Beta, and the Mac app is available now.

FAQ

What is the best LiteLLM alternative?

For a self-hosted open-source gateway, Bifrost is the closest in shape, and Portkey is the pick if guardrails matter most. Platform teams on Kubernetes should look at Agent Router. If you don't want to run anything, Cloudflare AI Gateway and Vercel AI Gateway are the managed options.

Was LiteLLM hacked?

Two malicious releases, 1.82.7 and 1.82.8, were uploaded to PyPI on March 24, 2026 and quarantined by PyPI. LiteLLM's security update says the official Docker image, LiteLLM Cloud and source installs were not affected, and a clean v1.83.0 shipped on March 30. Anyone who installed either bad version should rotate their credentials.

Is Envoy AI Gateway still maintained?

Yes, under a new name. It became Agent Router in September 2026 as an Agentic AI Foundation project. The project says the code, maintainers and Apache-2.0 licence are unchanged and old links redirect.

Is Portkey still open source after the Palo Alto Networks acquisition?

The gateway repository is still public and MIT-licensed. Palo Alto Networks closed the acquisition on May 29, 2026, and Portkey is now sold as Prisma AIRS AI Gateway. Check recent activity on the repository before you build on the open-source version.