// Software stack
Model gateway & cost control
One endpoint for every model: budgets and rate limits per team, automatic fallback, and clear spend per person and project.
5 pieces4 setup stepsscenario: Control AI costs and usage limits
What goes in it
- 1 ★ 62k
litellm Model gateways, routing & usage Open source
Self-hosted OpenAI-compatible gateway with budgets and limits
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
- 2
OpenRouter optional Model gateways, routing & usage Closed / commercial
Hosted multi-model routing, pay as you go
Unified API gateway to hundreds of models with pay-as-you-go billing.
- 3 ★ 13k
Portkey-AI/gateway optional Model gateways, routing & usage Open source
Open-source gateway with retries, caching and fallbacks
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
- 4 ★ 6.2k
helicone Observability & tracing Open source
Request logging and cost analytics
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
- 5
ccusage Model gateways, routing & usage Open spec
Usage stats for coding-agent subscriptions
How to set it up
- 01Decide hosted (OpenRouter: pay as you go, no ops) or self-hosted (LiteLLM / Portkey)
- 02Issue a virtual key per person or project with a monthly budget and per-minute rate limit
- 03Configure fallbacks to a backup model on timeouts or rate limits
- 04Track spend per person and project with Helicone or the gateway dashboard; use ccusage for coding-agent subscriptions
✓ Good fit
- Many people and projects share API keys and the bill is opaque
✗ Look elsewhere if
- A single person on one subscription
Similar stacks: what’s different
- Multi-model gatewayThat is a short list of trending tools; this gives the full budget, rate-limit and fallback setup