Microsoft plans huge deployment of its own AI chips, with an eye on saving costs 

Microsoft Announces Massive Rollout of Maia 300 AI Chips to Boost Cloud Efficiency

Microsoft Unveils Next-Generation Maia 300 AI Chip for Enhanced Cloud Performance

By Stephen Nellis and Max A. Cherney

Maia 300: Third Generation Custom AI Chip

SAN FRANCISCO, Oct 12 (Reuters) - Microsoft announced the next generation of its custom AI chip on Monday and said it plans to deploy potentially more than a million of them next year to back its internal AI services.

The Maia 300, as the chip is called, is the third generation of the company's custom chips designed to handle AI data crunching such as coding and carrying out complex business tasks on a user's behalf. Its predecessor is already running Microsoft's in-house AI models, as well as models from OpenAI delivered on Microsoft's Azure cloud computing service.

Industry Context: Competing with Nvidia and Other AI Firms

The new version comes amid a flurry of custom chips from both cloud computing rivals and AI firms such as OpenAI aimed at loosening the hold of Nvidia on the AI computing market.

Technical Innovations in Maia 300

Chiplet Architecture and Advanced Manufacturing

To construct the chip, Microsoft took a different technical path from Nvidia whose AI accelerators use one massive computing chip surrounded by high-bandwidth memory chips.

The central computing processor in Microsoft's Maia 300 silicon is composed of "chiplets"  —   typically specialized chips bonded together  —  that use 2-nanometer technology from Taiwan Semiconductor Manufacturing Co for key computing functions.

Supporting Chiplets and System Design

Other chiplets included in the design use cheaper 3-nanometer tech for functions such as moving data off the processor and elsewhere inside the system.

Performance and Efficiency Metrics

The Maia 300 processors will go into racks that contain 96 chips and use about 167 kilowatts of power, compared with the 142 kilowatts used by Nvidia's most recent GB300 AI server racks with 72 of its graphics processing units (GPUs).

Microsoft said the Maia 300 chips will get about 40% more useful computing done per megawatt than "leading GPU-based systems," though it did not name the systems it used for a comparison.

Strategic Impact and Future Plans

System Optimization and Executive Insights

"It's basically delivering more intelligence for every watt, every dollar, and every byte of memory by optimizing across the entire system," Rani Borkar, president of Azure hardware systems and infrastructure at Microsoft, told Reuters in an interview.

Deployment Scale and Capital Expenditure

Microsoft said it plans to roll out "multi-gigawatt deployments" of the Maia 300 chip starting next year. Microsoft has said it expects to spend more than $175 billion in capital expenditures this calendar year.

(Reporting by Stephen Nellis and Max Cherney in San Francisco; Editing by Sanjeev Miglani)