Running two concurrent Llama 3.1 8B inference services on a single NVIDIA H100 GPU using Kubernetes Dynamic Resource Allocation (DRA) and NVIDIA Multi-Instance GPU (MIG) technology on Red Hat OpenShift 4.21. The guide covers enabling the DRAPartitionableDevices alpha feature gate, installing Node Feature Discovery, NVIDIA GPU Operator, and the NVIDIA DRA driver, then deploying two vLLM model servers via the llm-d framework with Kustomize overlays. MIG partitions the H100 into two 3g.40gb slices (40 GB each), while DRA's shared counter mechanism automatically assigns non-overlapping memory slices to each pod, providing hardware-level isolation. The result is two fully isolated LLM services sharing one GPU, roughly doubling infrastructure efficiency compared to traditional all-or-nothing GPU allocation.
