MLflow AI Gateway now supports configurable guardrails that enforce content policies at the gateway layer, before requests reach LLMs or responses reach users. Three built-in types are available: Safety (toxicity filtering), PII Detection, and Custom. Guardrails can be set to either block requests with an HTTP 400 error or sanitize (redact) offending content. They run in order per endpoint, support before/after pipeline stages, and use a separate LLM judge model for evaluation. Configuration is done through the MLflow UI with no application code changes required.
Table of contents
How Guardrails Work Built-in Types Creating a Guardrail What Blocking Looks Like Managing Guardrails Getting Started 431 Impressions