> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

---
title: "AI Hardware & Accelerators"
url: https://daily.dev/agentic-ai-hub/ai-hardware-accelerators/
description: "The 2026 picture is that NVIDIA still dominates training and general inference."
lastUpdated: "2026-07-22"
---

The 2026 picture is that NVIDIA still dominates training and general inference. A growing field of custom silicon (TPUs, Trainium, in-house hyperscaler chips) and inference-specialist startups (Groq, Cerebras, and transformer-ASIC entrants like Etched) chip away at specific workloads.

## NVIDIA: Blackwell / GB-series

**Blackwell** is NVIDIA's current-generation datacenter architecture. Key parts (as of July 2026):

- **B200 / GB200.** The Blackwell workhorses. **GB200 NVL72** is a rack-scale system linking 72 Blackwell GPUs and 36 Grace CPUs over NVLink into one liquid-cooled unit that acts as a giant GPU. [https://www.nvidia.com/en-us/data-center/gb200-nvl72/](https://www.nvidia.com/en-us/data-center/gb200-nvl72/)  
- **B300 / Blackwell Ultra.** A mid-generation refresh that began shipping in late 2025. Per NVIDIA/vendor materials roughly **288 GB HBM3e** per GPU, higher FP4 throughput than B200, ~1,400W, liquid-cooled. **GB300 NVL72** is the rack-scale version. Emphasis on low-precision (**FP4**) inference.  
- **Vera Rubin (R100).** The next architecture, first unveiled at GTC 2025 and detailed further through CES/GTC 2026, scheduled for release in **H2 2026** (around Q3), and expected to pressure Blackwell pricing over time.

NVIDIA's real moat is **CUDA + NVLink + the whole software stack** as much as the silicon.

## Google TPU

**Ironwood** is Google's **seventh-generation TPU** (aka TPU v7 / "TPU7x"), positioned as inference-first and reaching general availability in 2026. Google cites large per-chip gains over prior generations and superpods of up to **9,216 chips** with massive shared HBM and Optical Circuit Switching. TPUs are available on Google Cloud (and power Gemini). The tradeoff is a **JAX/XLA-centric software path** vs. CUDA. [https://cloud.google.com/tpu/docs](https://cloud.google.com/tpu/docs)

## AMD

**Instinct MI300X/MI325X** (HBM-heavy, competitive memory capacity per GPU) established AMD as the credible #2. The **MI350 series (CDNA 4)** is the current-gen competitor to Blackwell, with the **MI400 series** slated for 2026 and rack-scale "Helios" systems positioned against NVIDIA's NVL racks. AMD's software stack is **ROCm**. Support has improved a lot (vLLM, PyTorch) but remains less turnkey than CUDA for edge cases.

## AWS Trainium & other hyperscaler silicon

**AWS Trainium** (Trainium 2 in wide use; **Trainium 3** now generally available via Trn3 UltraServers) targets cost-efficient training/inference on AWS via the Neuron SDK. Also in the field: **Microsoft Maia**, **Meta MTIA**, and Intel **Gaudi**, mostly captive to their clouds/workloads. Outside the US ecosystem, **Huawei Ascend** (910C, and the rack-scale CloudMatrix 384 supernode) is the dominant non-NVIDIA training/inference platform, especially in China, and is benchmarked directly against GB200 NVL72. → [hiascend.com](https://www.hiascend.com/en)

## Inference-first chips

| Chip | What it is | Watch out |
| :---- | :---- | :---- |
| [Groq](https://groq.com/) | Deterministic **LPU** built for very low-latency token generation; very high tokens/sec on served models | Not a training chip; consumed as an API/cloud with a curated model menu |
| [Cerebras](https://www.cerebras.ai/) | **Wafer-scale** engine (a whole wafer as one chip); large on-chip memory removes the bandwidth wall for high inference speeds, and it trains too | Specialized; accessed via Cerebras cloud/appliances |
| [Etched (Sohu)](https://www.etched.com/) | A **transformer-specialized ASIC** that bakes the architecture into silicon for very high throughput and low cost on transformer inference; emerged from stealth in 2026 with large reported pre-orders | Runs only transformer models (a bet the architecture stays stable), and it's early 🔄 |
| [SambaNova](https://sambanova.ai/) | Reconfigurable **RDU** (SN40L) serving very fast tokens via SambaNova Cloud; a Groq/Cerebras peer | Consumed as a cloud/appliance; curated model menu |
| [d-Matrix (Corsair)](https://www.d-matrix.ai/) | **In-memory-compute** inference accelerator, in full production in 2026, aimed at high-throughput low-cost serving | Early; specialized inference-only silicon 🔄 |

## VRAM math & cost-per-token intuition

- **Can I even load it?** Weights ≈ `bytes-per-param × params`: FP16 = 2 (7B ≈ 14 GB), INT8 = 1 (≈ 7 GB), INT4 = 0.5 (≈ 3.5 GB). Then add **KV cache** (grows with batch × context) and overhead. A 70B model in 4-bit is ~35 GB of weights alone, so it needs a 48 GB card (tight) or two.  
- **Cost per token** is set by throughput, not sticker price: `$/token ≈ (GPU $/hour) ÷ (tokens/hour)`. A pricier GPU that serves 3× the tokens can be *cheaper* per token. This is why quantization, batching, and the right serving engine ([Inference, Serving & Optimization](/agentic-ai-hub/inference-serving-optimization/)) move the economics more than the hardware line item.  
- **Cloud vs own:** rent (per-hour or per-token APIs) for spiky/uncertain demand and to skip capex and ops. Buy/reserve when utilization is high and steady (owned/reserved GPUs amortize well below on-demand at ~24/7 use). Most teams should rent until utilization and unit economics clearly justify owning.