Google engineers present a TPU microbenchmark suite for empirically evaluating accelerator performance across five core areas: compute, memory (HBM), network (ICI), host transfer, and collective operations. The suite uses the Roofline model to classify workload bottlenecks as compute-bound or memory-bound, enabling targeted optimization. A case study on a 110B Mixture-of-Experts training workload on a 4x4x4 TPU 7x (Ironwood) configuration demonstrates how microbenchmarks guide kernel and mesh optimizations. Key insights include how the 256x256 systolic array architecture constrains operand shapes and why head_dim alignment to 256-byte boundaries matters for MXU utilization. The open-source suite is available on GitHub and supports predictive modeling of large-scale deployments without running expensive full-slice tests.
