Published on November 16, 2023, 4:11 pm

Microsoft Unveils Custom AI Chip and CPU for Cloud Infrastructure

In a move to reduce reliance on Nvidia GPUs and prepare for increased AI usage, Microsoft has developed its own custom AI chip called Azure Maia and an Arm-based CPU named Azure Cobalt. These chips are set to arrive in 2024 and will be used to optimize Microsoft’s cloud infrastructure.

The Azure Cobalt CPU is a powerful 128-core chip designed for general cloud services on the Azure platform. On the other hand, the Maia 100 AI accelerator is specifically intended for cloud AI workloads such as training and inferencing large language models.

Currently, the Maia chip is being tested on GPT-3.5 Turbo, which powers various Microsoft products including the free version of ChatGPT, Bing AI, and GitHub Copilot. This shows Microsoft’s commitment to enhancing the performance of its AI-driven technologies.

To complement these new chips, Microsoft also introduced Azure Boost—a system that offloads storage and networking processes from host servers to purpose-built hardware and software. By developing their own AI chip and CPU, Microsoft aims to control costs more effectively while optimizing the performance of their cloud infrastructure.

Although specifications and performance comparisons have not been released yet, Microsoft CEO Satya Nadella claims that the Cobalt 100 CPU is “the fastest of any cloud provider.” Furthermore, it is said to be 40% faster than the commercial Arm-based servers currently in use by Microsoft.

It’s important to note that despite these advancements in AI hardware, Microsoft will continue to utilize AMD’s latest MD Instinct MI300X GPUs as well as Nvidia’s NVIDIA H200 Tensor Core GPU. Nadella emphasizes that these new chips are complementary rather than a replacement for existing technology partnerships.

In addition to announcing these innovative chips, Nadella shared insights about Copilot—an AI-powered chatbot developed by Microsoft. According to research conducted by Microsoft in its Work Trend Index, 70% of Copilot users reported increased productivity, while 68% said it improved the quality of their work.

Given the popularity and positive feedback from users, Microsoft is expanding its Copilot offerings. Bing Chat has been rebranded as Copilot, aimed at competing with OpenAI’s ChatGPT. This move positions Copilot as a free AI chatbot option, while a paid version called Copilot for Microsoft 365 will be available for enhanced features.

Microsoft also unveiled Copilot Studio, a low-code solution that enables organizations to build custom AI chatbots and copilots. This extends the capabilities of Microsoft 365 Copilot by providing customization options for data sets, automation flows, and non-Microsoft Graph copilots.

But that’s not all—Copilot for Service promises to accelerate the AI transformation in customer service by incorporating generative AI into customer contact centers. Agents can ask natural language questions to Copilot for Service and receive insights based on knowledge repositories.

In Microsoft Dynamics 365, Copilot will integrate generative AI and mixed reality with augmented reality (AR) service applications initially focused on Hololens 2. Additionally, Copilot for Azure aims to simplify IT management by understanding user roles and goals, enhancing their ability to design, operate, and maintain applications and infrastructure components.

Copilot for all is projected to be available starting from December 1st, 2023. The introduction of these new chips along with the expansion of the Copilot ecosystem demonstrates Microsoft’s commitment to advancing artificial intelligence technologies across various industries.


Comments are closed.