IEEE Spectrum: 2026 Is the Inference Hardware Turning Point as GPUs Sit Idle 50-80% of the Decode Phase

Matthew Smith argues inference has replaced training as the industry's design center, quoting analyst Matt Kimball that 'training is yesterday's news.' The piece benchmarks four competing bandwidth attacks: d-Matrix stacks compute on DRAM, Majestic Labs extends the memory interface up to a meter to reach 128TB per rack, Nvidia's Groq 3 uses 500MB of SRAM at seven times GPU bandwidth, and Cerebras keeps 44GB of on-chip SRAM on its wafer-scale engines.