LiteLLM is an open-source AI gateway that proxies all LLM requests through a single endpoint, providing unified monitoring, access control, and cost tracking across providers like OpenAI and Anthropic. The post walks through running LiteLLM locally with Docker and PostgreSQL, configuring it via YAML, writing a demo Python client, integrating OpenTelemetry traces with VictoriaTraces, and exploring access management features including teams, users, API keys, budgets, and rate limits. Key span attributes for cost, token usage, model info, and user identity are highlighted. The author notes that while the monitoring and tracing capabilities are impressive, user management has some confusing behaviors — particularly around keys created outside a team bypassing team-level rate limits.
