On October 12, Microsoft unveiled Maia 300, an in-house inference accelerator it wants to make the main engine behind its AI products. The promise is 40% more tokens per megawatt than leading GPU systems; the fine print is that the company didn't publish what baseline that is measured against.
According to CNBC, Jon Peddie Research, and Interesting Engineering, the Maia 300 rack design houses 96 accelerators in a 167 kW envelope. For comparison, Nvidia's GB300 racks have 72 GPUs and draw around 142 kW. The performance figures come from Microsoft, not independent testing.
Tokens per megawatt is the right metric for inference: a data center's limit today is power, not space. But the figure comes with no baseline: Microsoft did not identify which GPU system it compared against. Also, the Maia rack draws more total power (167 kW versus ~142 kW), so the advantage, if it exists, would come from more work per watt rather than lower consumption. Until independent benchmarks appear, it is a design target, not a fact.
The sources place Maia 300's deployment in 2027, with GitHub Copilot and Microsoft Foundry workloads planned to follow toward the end of 2026. Microsoft gave no precise production date and did not promise lower prices for customers. Nor is there a figure for how much its Nvidia purchases would fall: for training, Microsoft continues to work with Nvidia and AMD.
Maia joins other hyperscaler in-house accelerators; we already covered Meta and its MTIA chip. The logic is the same: whoever runs inference at scale looks to cut cost per token with custom silicon, leaving Nvidia GPUs for training and workloads where the CUDA ecosystem matters most.
Carlos Montiel is an enterprise AI solutions architect. He implements LLMs, Agents, RAG, and orchestrators for companies across Guatemala and Latin America. Reach out for a consultation.
Contact Carlos Montiel