Published on November 16, 2023, 4:58 pm
Microsoft has announced the development of two custom-designed AI chips, Azure Maia 100 AI Accelerator and Azure Cobalt 100 CPU, to support its AI innovation. The company aims to optimize and integrate every layer of the infrastructure stack for maximum performance and to diversify its supply chain.
The scarcity of GPUs has led Microsoft and other companies in the AI space to rely heavily on chip vendors like Nvidia. However, the shortage of AI GPUs could extend into 2025, according to TSMC CEO. To address this challenge, Microsoft has taken it upon itself to develop its own in-house AI chips.
The Maia 100 chip is specifically designed for the Azure hardware stack and can power large internal AI workloads running on Microsoft Azure. It contains 105 billion transistors and is engineered to achieve maximum hardware utilization. While not many technical details have been provided about Maia 100, it is known that its physical package is larger than a typical GPU’s.
Cobalt 100, on the other hand, is an energy-efficient chip built on an Arm Neoverse CSS architecture. It will deliver greater efficiency and performance in cloud native offerings. Microsoft plans to use Cobalt 100 to power new virtual machines for customers in the coming year.
The collaboration between Microsoft and OpenAI played a crucial role in the development of these custom AI chips. OpenAI provided feedback on Maia 100’s design, enabling Microsoft to optimize its infrastructure down to the silicon level. This partnership emphasizes both companies’ commitment to making capable models more affordable for customers.
Developing their own AI chips allows Microsoft to be more self-reliant and cost-conscious in the competitive field of AI dominance. The company believes that by optimizing every layer of its technology stack, it can provide customers with better options for performance, cost, and other dimensions they care about.
However, creating custom AI chips is not without challenges, as seen with Meta’s early chip efforts and Google’s struggles to meet demand for its TPUs. Microsoft aims to overcome these obstacles by prioritizing performance, power efficiency, and cost in its chip designs.