Microsoft has unveiled Maia 300, a custom accelerator it intends to make the primary inference system for its own products and services as large deployments begin in 2027. Microsoft says the third-generation chip and its surrounding rack system can deliver up to seven times the token throughput and more than 2.5 times the performance per watt of Maia 200. The larger change is strategic: a company that spends heavily on Nvidia hardware now wants much more control over the machines that produce every Copilot answer.
Inference is the part of AI computing that runs a trained model and generates an answer. It is increasingly expensive as products add longer context, reasoning steps and tool-using agents. Microsoft is attacking that bill as a systems problem rather than a chip-only benchmark. Maia 300 combines custom silicon, memory, networking, cooling and scheduling into one design built around the work Microsoft expects to run at cloud scale.
The rack is the product
Each Maia 300 accelerator uses four compute chiplets made on TSMC’s N2P process, plus separate components for memory and input-output work. Eight HBM4 stacks provide 288 GB of memory and 18 TB per second of bandwidth. Microsoft says this gives the chip 2.5 times Maia 200’s memory bandwidth, while new low-precision formats and explicit control of data placement reduce the energy spent moving model state around.
The company is also stretching the accelerator beyond one server. A rack holds 96 Maia chips and draws up to 167 kilowatts. A two-tier Ethernet-based fabric can join 128 racks into a 12,288-accelerator superpod, with optical links handling the longer paths. The design lets software keep latency-sensitive work nearby, move model state directly between prefill and decoding workers, and use spare capacity in another rack when hardware fails.
That scale explains why the announcement matters more than another impressive rectangle of silicon. Microsoft is building a vertically integrated inference fleet whose economics can be tuned from arithmetic precision down to water loops and power feeds. Axios reports that Microsoft expects Maia to produce 40 percent more tokens per megawatt than leading GPU-based systems and sees inference as its largest and fastest-growing AI-computing opportunity.
The benchmark still belongs to Microsoft
The biggest limitation is that the boldest numbers are vendor claims against a previous Microsoft chip or an unnamed GPU system. Microsoft has not supplied an independently reproducible comparison covering model quality, latency, software maturity, acquisition cost and utilization. Seven times the throughput sounds decisive until the denominator, workloads and deployment assumptions are fully visible.
The transition also starts next year and will take longer than 2027 to reach the promised multi-gigawatt scale. Microsoft says Maia will become primary for its own services, not that Azure customers or OpenAI will suddenly abandon Nvidia. Even successful custom chips still depend on constrained TSMC manufacturing, advanced packaging and high-bandwidth memory. Owning the blueprint does not conjure a supply chain.
TINA’s view: custom silicon is becoming cloud policy
TINA’s view: Maia 300 is less a declaration that Nvidia has lost than evidence that hyperscalers no longer want one supplier setting the economics of every generated token. Microsoft can use its own workloads to improve hardware, software and scheduling together, then keep the savings inside products where customers may never know which accelerator answered them. The chip is therefore leverage even before it wins an open benchmark.
That judgment would weaken if Maia deployments remain small, slip materially beyond 2027 or require Nvidia systems to handle the difficult workloads Microsoft claims its design targets. It would strengthen if Microsoft publishes comparable production results and moves Copilot, GitHub Copilot and Foundry inference onto Maia without sacrificing reliability or model choice.
The signal to watch is utilization, not launch-day peak performance: how much of Microsoft’s real inference traffic moves to Maia, at what cost per useful token, and whether outside Azure customers can eventually choose it. If those numbers arrive, Nvidia will still have an enormous business. It will also have a customer that has become a serious systems rival.
This article was produced by TINA, TechInform’s AI editorial system, using linked public sources. The hero is an original AI-generated editorial illustration.
