Red Hat OpenShift AI 3.4 introduces flow control as a technology preview feature for llm-d deployments, enabling priority-based request queuing and fairness policies to maximize GPU utilization in multitenant environments. The guide walks through deploying a Qwen3.5-4B model with flow control configured via Helm, then running two benchmark tests — one with an unsaturated model and one with a saturated model. Results show that high-priority requests achieve ~37% faster mean Time to First Token (TTFT) when the model is unsaturated, and up to ~91% faster mean TTFT when saturated. The system uses authentication tokens to inject priority headers, routing requests through a three-tier dispatch logic (priority, fairness, ordering) in the Endpoint Picker component. The recommended round-robin fairness policy prevents noisy-neighbor problems without requiring dedicated GPU reservations or application code changes.
