Hackaday

Hacker News

XDA Developers

Two posts this week are making the same case from different angles: old datacenter GPUs, specifically the NVIDIA Tesla V100 32GB, are quietly becoming the budget option for running large models locally.
The pitch is real. A used V100 PCIe 32GB runs $400-500 from overseas sellers. A used RTX 3090 costs $700-1,050 and tops out at 24GB. If you want to run Gemma 4 26B or Qwen3 35B entirely in GPU memory without paying cloud prices, the V100 is currently the cheapest path past that 24GB ceiling. One builder went further and wired two V100s onto a custom PCIe card, getting 64GB of GPU memory in a desktop chassis.
But the caveats are piling up, and they’re not small.
CUDA Toolkit 13.0 just dropped Volta support. PyTorch is moving to deprecate compute capability 7.0. LM Studio doesn’t support it at all. Prebuilt llama.cpp binaries no longer include Volta, so you’re compiling from source. The card has no BF16 or FP8 support, which matters for modern quantization formats. It’s passively cooled, meaning you need to mod your airflow or it throttles. And there’s no display output, so it’s inference-only.
This is the classic enthusiast tradeoff: you’re buying hardware the datacenter world has already moved past, which is exactly why it’s cheap, and exactly why the software ecosystem is quietly walking away from it.
For anyone who wants to just install something and have it work, the RTX 3090 is still the answer. The V100 is for people who are comfortable building llama.cpp from source, don’t mind the passive cooling constraints, and genuinely need that extra 8GB of headroom. That’s a narrower audience than the price tag makes it look.
The interesting meta-story here is that “run LLMs locally” has gotten complicated enough that the hardware choice now comes with a full software compatibility matrix attached. A year ago the conversation was about whether local inference was even viable. Now it’s about which deprecated datacenter card has the best VRAM-per-dollar ratio and whether you can still compile the right binaries for it.

Trends

Used NVIDIA Tesla V100 32GB datacenter GPUs ($400-500) are emerging as the cheapest way to exceed the 24GB VRAM ceiling for running large models locally, undercutting the RTX 3090 ($700-1050). However, significant caveats exist: CUDA Toolkit 13.0 dropped Volta support, PyTorch is deprecating compute capability 7.0, LM Studio doesn't support it, prebuilt llama.cpp binaries no longer include Volta, and there's no BF16/FP8 support. The card is passively cooled and has no display output. The V100 suits only technically capable users comfortable compiling from source who genuinely need the extra VRAM headroom — the RTX 3090 remains the practical choice for most. The broader trend: local LLM hardware selection now comes with a full software compatibility matrix.

The V100 datacenter GPU is the cheapest way past 24GB VRAM, if you can stomach the pain

Been looking at getting one of these off of ebay but the old architecture and lack of support has been putting me off. I’ll stick to my 4090 for now but would be cool to play around with something more powerful locally.

I purchased a V100 16gb model, I have it running in a headless server at home, so not using the latest drivers and code base. It’s a nice gimmick, but I am undecided on getting more of them, I was thinking of gettign 4 of the 16gb models since the 32gb models are like 4 times the cost of the 16gb but I doubt that will happen