# LiteLLM LiteLLM is an [[Open Source]] [[AI Gateway]] from BerriAI that puts a single OpenAI-compatible interface in front of 100+ models. It comes in two shapes, and the difference matters more than the docs make it sound. **The Python SDK** is a library you import. Your code calls `completion()` and LiteLLM handles the provider-specific translation, retries and fallbacks in-process. **The proxy server** is a standalone gateway you run yourself, usually as a [[Docker]] container. It adds authentication, virtual keys, per-project budgets and spend tracking across a whole team. See [[LiteLLM Proxy Configuration]] for the `config.yaml` details. ## Why I keep coming back to it Every hosted gateway asks you to accept one more company between your prompts and the model. LiteLLM does not. You run it on your own machine or your own cluster, and the traffic never leaves infrastructure you control. For gateways it plays the same role Ollama plays for inference: the escape hatch when the privacy question has no acceptable hosted answer. It is also the reason a lot of odd setups work at all. Pointing [[Claude Code]] at a completely different provider, or at a subscription you already pay for, usually comes down to putting LiteLLM in the middle and letting it translate. ## The trade-offs - You now operate a piece of infrastructure. It needs a database for virtual keys and spend tracking, plus upgrades and monitoring. - The advanced [[AI Guardrails]], SSO and audit logs sit in the paid enterprise tier. - Provider coverage is broad, so quality varies. Test the specific model and provider pair you care about instead of trusting the matrix. ## References - Documentation: https://docs.litellm.ai/ - Proxy server docs: https://docs.litellm.ai/docs/simple_proxy - Repository: https://github.com/BerriAI/litellm - Releases and changelog: https://github.com/BerriAI/litellm/releases - Enterprise tier and pricing: https://www.litellm.ai/enterprise ## Related - [[LiteLLM Proxy Configuration]] - [[AI Gateway]] - [[Model routing]] - [[Open Source]] - [[Docker]] - [[Claude Code]] - [[AI Observability]] - [[AI Guardrails]] - [[OpenRouter]]