A detailed guide to building a budget multi-GPU LLM inference rig using four AMD RX 580 8GB cards sourced from crypto-mining liquidation sales for $40–$60 each. The total build costs under $500 and runs quantized GGUF models via llama.cpp. Covers hardware selection and gotchas (ex-mining cards, PSU sizing, BIOS settings), two software paths—ROCm/HIP with the HSA_OVERRIDE_GFX_VERSION=8.0.3 workaround for unsupported Polaris GPUs, and the more reliable Mesa Vulkan fallback—plus compiling llama.cpp for AMD multi-GPU inference, layer splitting with --tensor-split, realistic performance benchmarks (8–15 t/s for 7B models, 2–5 t/s for 70B across four cards), undervolting tips, and an honest assessment of limitations including PCIe x1 bandwidth bottlenecks and driver fragility.
