Microsoft Maia 300 Packs 288 GB HBM4, 7x Maia 200 Throughput

Microsoft's third-generation in-house accelerator, Maia 300, pairs TSMC N2P compute chiplets with eight HBM4 stacks for 288 GB of capacity and 18 TB/s of bandwidth per chip. Ninety-six of them sit inside a 167 kW liquid-cooled rack, double the accelerator density of the previous Maia 200.

In a deep-dive post on the Azure blog, Microsoft's Rani Borkar and Saurabh Dighe write that the setup delivers "up to 7 times greater rack-scale token throughput in its full 96-accelerator configuration and more than 2.5 times the performance per watt" versus Maia 200. The stated design goal is to "optimize the entire system for LLM inference so that every watt and every dollar delivers more intelligence."

The silicon is a multi-chiplet design. Compute dies ride N2P, while memory-controller and I/O logic sit on separate dies chosen for bandwidth and cost. Layered on top is what Microsoft calls a Software Defined Locally Accessed Dataflow Architecture, giving compilers explicit control over data placement, DMA engines, and synchronization across local memories, compute engines and the network.

At rack scale, 24 four-accelerator trays interconnect electrically inside the rack; an optical second tier can stitch up to 12,288 accelerators into a single superpod across 128 racks, with up to 750 GB/s of cross-rack bandwidth per chip. Liquid cooling absorbs more than 80% of the thermal load. Microsoft names Copilot, GitHub Copilot and Microsoft Foundry as the kind of workloads the system is built around.

The post is architecture, not roadmap. It does not name a production volume, a deployment target in gigawatts, or which products get Maia 300 silicon first — gaps that stand out against the 457 AI infrastructure stories we've tracked over the last 90 days, most of which came attached to a build-out number.