LLM Key Server: Providing Secure and Convenient Access to Internal LLM APIs

Mercari's AI Security team built an LLM Key Server to provide secure access to internal LLM APIs using temporary credentials instead of static API keys. The system leverages LiteLLM for unified API access and Google's OIDC ID tokens for authentication, issuing short-lived keys that automatically renew. It supports multiple environments including local development, GitHub Actions, and Google Apps Script through common templates, eliminating manual key management while reducing security risks from leaked credentials.