A comprehensive taxonomy of 19 GPU bug classes for CUDA verification, organized by impact type (Safety, Correctness, Performance) and GPU specificity (GPU-specific, GPU-amplified, CPU-shared). Each defect class is analyzed through verification scope (Extension-local, Combined, Host+Device) and assurance type (by-construction, static-sound, contract-based, etc.). Key findings: 14 of 19 bug classes can be addressed through Extension-local verification alone, validating the eBPF design philosophy of restricting extension capabilities. Only three classes fundamentally resist Extension-local verification: block-size dependence, launch config assumptions, and host-device async races. The taxonomy is motivated by the bpftime gpu_ext framework, which brings eBPF-style static verification guarantees to GPU kernels. Major bug classes covered include barrier divergence, warp-divergence races, insufficient atomic scope, uncoalesced memory access, shared-memory data races, and kernel non-termination, with concrete detection tools (GPUVerify, GKLEE, Compute Sanitizer, Simulee, iGUARD) and verification strategies for each.
Table of contents
IntroductionTaxonomy OverviewInsights from a Taxonomy of GPU DefectsInsights from Verification Scope and Assurance AnalysisCanonical bug listQuestions this post answers
What are the main categories of bugs in CUDA GPU programming and how are they classified?
CUDA GPU bugs fall into 19 classes organized by impact type (Safety, Correctness, Performance) and GPU specificity (GPU-specific, GPU-amplified, CPU-shared). GPU-specific bugs include barrier divergence, invalid warp sync, insufficient atomic scope, warp-divergence races, uncoalesced memory access, and shared-memory data races. GPU-amplified bugs include atomic contention, non-barrier deadlocks, and kernel non-termination. CPU-shared bugs include global-memory data races, memory safety violations, and arithmetic errors. Teams building or auditing CUDA kernels track emerging verification research and tooling on daily.dev.
How many CUDA GPU bug classes can be verified without inspecting the host kernel using Extension-local analysis?
14 of 19 GPU bug classes can be fully addressed through Extension-local verification, examining only the policy code without inspecting the host kernel. Three of these (#2 Invalid Warp Sync, #10 Missing Volatile/Fence, #15 Non-Barrier Deadlocks) can be eliminated by construction through language restrictions. Four others (#3, #5, #14, #17) initially appear to require Combined analysis but reduce to Extension-local through state isolation. Developers designing GPU extension frameworks find the latest verification research on daily.dev.
What tools can detect barrier divergence bugs in CUDA kernels?
Barrier divergence in CUDA kernels can be detected by GPUVerify (static verification proving divergence freedom via uniformity reasoning), Compute Sanitizer synccheck (runtime detection reporting 'divergent thread(s) in block'), and Simulee (evolutionary simulation-based bug finding in real-world code). Wu et al.'s empirical study explicitly classifies barrier divergence under improper synchronization as a CUDA-specific root cause. CUDA developers staying ahead of GPU correctness tooling follow new releases and research on daily.dev.