// Switch models, control cost
litellmv1.104.2
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Open source Maintained For agents
IN ONE PARAGRAPH
litellm is the fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM].
Good for: Developers who want their agent (Claude Code, Codex, Cursor…) to switch models, control cost.
01How to start
- 1Read the official README for requirements and permissions.
- 2Install it from the official package registry:
pip install litellm - 3Tell your agent the command exists (e.g. in AGENTS.md / CLAUDE.md) so it knows when to call it.
02Alternatives & comparison
Compare side by side → Portkey-AI/gateway A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. ★ 13k bifrost Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS. ★ 9.0k any-llm Communicate with an LLM provider using a single interface ★ 2.2k plano Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic. ★ 7.1k ai-gateway The fastest, lightest, and easiest-to-integrate AI gateway on the market. Fully open-sourced. ★ 636
03Related
- ?Use Claude Code with other models (Chinese models, your own gateway)
- ?Use Codex CLI with local or custom model providers
- ?How much have Claude Code / Codex used and cost?
- ?Set budgets and rate limits for a team’s LLM usage
- ?Where to find free LLM APIs, and what the conditions are
- ?Are API relays / OpenRouter safe?
- ?LiteLLM 1.107.0 was compromised: am I affected, and what to do
- ?Self-host Dify on company servers (Docker Compose)
- ?Employees paste company data into AI: what to do
- +Model gateway & cost control · Self-hosted OpenAI-compatible gateway with budgets and limits5
- +Enterprise RAG knowledge base · One gateway to many models, with cost and rate limits6
- ›Ollama 文档 · OllamaEnglish
- ›handy-ollama 动手学 Ollama · DatawhaleChinese
- ›vLLM 文档 · vLLMEnglish
- ›SGLang 文档 · SGLangEnglish
- ›llama.cpp · ggml-orgEnglish
- All tutorials →