“LLM gateway” is one label stretched over three different jobs. Read the source and they separate cleanly by what they route and who they’re for. CLIProxyAPI pools your personal subscription logins. LiteLLM routes your API keys with metering and failover, from inside your app. Portkey is a stateless edge gateway that routes by a declarative config you hand it per request. I read CLIProxyAPI at v7.2.47, LiteLLM at v1.92.0-dev.1, and Portkey at v1.15.2, and the sharpest way to tell them apart turns out to be a single question: what state does the gateway keep? Every claim points at a line.
The one question
A gateway sits between your code and the model providers. What separates these three is the unit they route and the state they hold to do it:
- Subscription accounts — pool the flat-fee logins you already pay for. (CLIProxyAPI)
- API keys / deployments — spread metered, pay-per-token keys across providers for reliability. (LiteLLM)
- Any request — a stateless front door that translates and routes to whatever provider a config names. (Portkey)
Camp 1 — Pool your subscriptions: CLIProxyAPI
CLIProxyAPI exists to squeeze more out of the ChatGPT/Claude/Gemini subscriptions you already pay a flat fee for. You log in through it via OAuth, and it fans requests across your logged-in accounts. As I found in its teardown, it can’t see your quota — subscription limits are opaque — so it’s purely reactive: an account is used until a provider returns 429, then it’s benched until a cooldown timer expires (selector.go:305). The state it keeps is per-account cooldowns. This is the camp for an individual maxing out personal subscriptions — not for billing a team.
Camp 2 — Route your keys: LiteLLM
LiteLLM is a routing library (and optional proxy) for the API keys you pay per-token for. Its default is a weighted coin flip — simple-shuffle picks a random healthy deployment (simple_shuffle.py:59) — but its real leverage, as I found in its teardown, is that it meters every key: the smart strategies write per-deployment tpm/rpm/latency into a cache and can even reserve capacity before a call (lowest_tpm_rpm_v2.py:116), then fail over through retries, cooldowns, and fallbacks. The state it keeps is usage meters and cooldowns. This is the camp for a developer or app that wants reliability across paid providers, with the routing logic living in your process.
Camp 3 — Run a production gateway: Portkey
Portkey is the piece of infrastructure you stand up in front of everything. It’s a stateless Hono app (index.ts:46) that deploys to Cloudflare Workers or Node, exposes a unified OpenAI-compatible API (index.ts:147), and translates each request to one of ~70 providers through per-provider transformers (providers/index.ts:78). The defining move is that routing is data, not code: you pass a declarative config in the x-portkey-config header (globals.ts:18), and tryTargetsRecursively walks it (handlerUtils.ts:476), switching on strategy.mode — fallback (try targets in order), loadbalance (weighted random), single — and recursing into nested targets so you can express, say, load-balancing inside a fallback purely in JSON (handlerUtils.ts:662). It keeps no per-user meters in the routing path — a decision is a pure function of the config plus live response codes — which is exactly what lets it scale as horizontally as your worker count. On top it adds guardrail and cache plugins. This is the camp for a platform team fronting many apps and users.
The comparison
| CLIProxyAPI | LiteLLM | Portkey | |
|---|---|---|---|
| Routes | subscription OAuth accounts | API keys / deployments | any request, to ~70 providers |
| For | an individual pooling subscriptions | a developer / app | a platform team |
| Routing decided by | reactive cooldowns (in code) | metering + strategies | a declarative per-request config |
| State it keeps | per-account cooldowns | usage meters + cooldowns | none in the routing path |
| Sees cost/usage? | no (opaque quota) | yes (meters keys) | via response codes only |
| Ships as | a Go binary you self-host | a Python library / proxy | a Hono edge worker |
| Signature extra | session affinity | budgets & fallbacks | guardrails, cache, nestable config |
Why the split decides everything
The state column is the whole story. CLIProxyAPI must be reactive because it can’t see subscription quotas — so it keeps cooldown state and waits for the 429. LiteLLM can be proactive because it meters keys — so it keeps usage state and routes on it. Portkey deliberately keeps no routing state — so it can run as a swarm of identical stateless workers at the edge, and it pushes the routing decision out to a config you supply. Those aren’t three flavors of the same thing; they’re three different bets about where the intelligence and the state should live: in a reactive scheduler, in a metering library, or in a declarative front door.
Where do you fit?
- You personally pay for a few ChatGPT/Claude/Gemini subscriptions and want to pool them → CLIProxyAPI. Just know it’s guessing at your reset windows, not reading them (that’s its teardown’s punchline).
- You’re building an app and want reliable, metered routing across paid API keys, in-process → LiteLLM. Set a strategy and, ideally, Redis so the meter is shared; the default is only a weighted random.
- You’re a platform team putting a governed front door in front of many apps → Portkey. Declarative config, guardrails, caching, and stateless edge deployment are built for exactly that; the trade is that you operate a gateway.
Scope & honesty
I read CLIProxyAPI v7.2.47 (00114be), LiteLLM v1.92.0-dev.1 (88e03e5), and Portkey v1.15.2 (ca77129). The CLIProxyAPI and LiteLLM mechanisms were each verified against source in the linked teardowns; Portkey’s stateless-config-gateway shape was run through an adversarial claim-verifier at this commit and confirmed — including the stateless claim, which survived a hard look (its circuit-breaker and budget hooks are injected and never wired up in the open-source repo).
Two honesty notes. The camps describe each tool’s center of gravity, not a fence: LiteLLM also runs as a standalone proxy, and Portkey is self-hostable, so they overlap in the middle. And a number to keep straight — Portkey advertises “250+ LLMs,” but that’s a model count; the repo has about 70 provider integrations. With this, the three series I set out to analyze — coding agents, local inference, and gateways — each have their hub.
