# AI Gateway An AI gateway is a proxy layer between applications and AI model providers. It provides a unified API across multiple providers ([[Anthropic]], [[OpenAI]], [[Mistral AI]], [[Deepseek]], etc.), adding cross-cutting concerns like authentication, rate limiting, cost tracking, caching, logging, and [[Model routing]]. Key capabilities: - **Provider abstraction**: switch between models and providers without changing application code - **Fallback and retry**: automatically route to a backup provider if the primary is down or rate-limited - **Cost management**: set budgets, track spend per team/project/user - **[[Prompt caching]]**: cache repeated prompts to reduce latency and cost - **Observability**: log all requests/responses for [[AI Observability]] and debugging - **Guardrails**: apply [[AI Guardrails]] (content filtering, PII detection) at the gateway level [[OpenRouter]] is a well-known AI gateway that aggregates dozens of models behind a single API, with automatic routing and pricing optimization. Cloudflare AI Gateway, Portkey, and LiteLLM are other examples. For teams building [[AI Agents]] and [[Agentic Engineering]] workflows, a gateway reduces vendor lock-in, simplifies cost control, and provides a single point for observability and security. It's the AI equivalent of an API gateway in microservice architectures. ## References - ## Related - [[Model routing]] - [[OpenRouter]] - [[AI Observability]] - [[AI Guardrails]] - [[Large Language Models (LLMs)]] - [[Anthropic]] - [[OpenAI]] - [[AI Agents]]