A comprehensive review of profiling and tracing tools for GPUs, DPUs, and APUs. Covers NVIDIA's Nsight suite, AMD's ROCm toolchain (Omniperf, Omnitrace), Intel VTune, and open-source frameworks like HPCToolkit and RenderDoc. Discusses eBPF-based low-overhead approaches such as GPUprobe for production monitoring, DPU-specific telemetry challenges, and APU integrated profiling needs. Compares open-source vs. commercial solutions across feature depth, cross-vendor support, and extensibility. Highlights emerging trends including causal profiling, ML-driven analysis, unified multi-accelerator tracing, and standardized performance interfaces. Identifies persistent challenges: vendor fragmentation, black-box hardware, overhead trade-offs, and lack of cross-device coordination.
Table of contents
Profiling and Tracing Tools for GPUsProfiling and Tracing Tools for DPUs (SmartNICs)Profiling and Tracing Tools for APUs (CPU–GPU Integrated Platforms)Open-Source vs Commercial Solutions – A ComparisonWorkload-Specific Tool MappingsCurrent Research Directions and Emerging TrendsLimitations and Challenges of Existing ToolchainsConclusionReferencesQuestions this post answers
What is the overhead of NVIDIA Nsight Systems when profiling a CUDA application?
Nsight Systems typically slows programs by 2×–10× during a profiling session. It is designed for development and tuning rather than continuous or production use, because it must explicitly start a profiling session and collects detailed CPU-GPU timeline data via the CUDA Profiling Tools Interface (CUPTI). For always-on monitoring, lighter tools like GPUprobe are recommended instead. Engineers optimizing CUDA workloads track tool trade-offs like these on daily.dev.
What is GPUprobe and how does it monitor CUDA applications without instrumentation?
GPUprobe is a Linux eBPF-based GPU observability tool that uses uprobes to hook into NVIDIA's CUDA runtime library functions at the kernel level. It monitors events like cudaMalloc/free and kernel launches in real time without modifying the target application. Overhead is measured under 4% in benchmarks, making it suitable for continuous production use. It exports metrics in OpenMetrics format for Prometheus and Grafana dashboards. Teams running GPU services in production find continuous monitoring approaches like this on daily.dev.
What GPU profiling tools did AMD introduce in ROCm 6.2?
AMD introduced Omniperf and Omnitrace in ROCm 6.2. Omniperf is a kernel-level profiler for HPC and ML workloads on AMD Instinct GPUs, offering detailed hardware counter analysis via CLI or GUI dashboard. Omnitrace is a multi-purpose CPU and GPU profiling tool supporting dynamic binary instrumentation, call-stack sampling, and causal profiling for heterogeneous CPU-GPU applications. Both are open-source. Developers targeting AMD Instinct GPUs stay current on ROCm toolchain changes through daily.dev.