Nvidia unveils new AI chip H200

HGX H200 [Courtesy of Nvidia]

Nvidia, the forefront runner in AI chip manufacturing, has unveiled its newest product, the H200, designed specifically for high-performance data centers powering advanced artificial intelligence applications.

The company announced on Monday (local time) the release of HGX H200, a product tailored for training and deploying AI models at high-performance data centers.

The HGX H200 features the H200 tensor core graphics processing unit (GPU) paired with high-performance memory, HBM3e, with a massive capacity of 141GB. Media reports suggest that South Korea’s SK hynix is the supplier of the HBM3e memory.

According to Nvidia, the H200 offers nearly double the capacity and a 2.4x increase in bandwidth compared to one of its previous models, the A100. The new chip has inference speeds nearly twice as fast as the H100 based on a test using Meta’s open-source Llama 2 LLM.

Scheduled for release in the second quarter of 2024, Nvidia anticipates cloud service providers like Amazon Web Services (AWS), Google Cloud, Microsoft Azure, and Oracle Cloud to offer the H200 to their customers.

In the competitive landscape of AI semiconductor markets, Nvidia’s move with the H200 marks an effort to outpace competitors like AMD and emerging tech giants entering this sector. The demand for AI computing, fueled by AI models like ChatGPT, has led Nvidia to seize around 80 percent of the AI semiconductor market, driving substantial performance and stock price growth.

Meanwhile, OpenAI’s CEO, Sam Altman, shared views in an interview with the Financial Times predicting a changing scenario. While acknowledging Nvidia’s past supply issues with the H100, Altman expects a more favorable situation next year. He anticipates decreased dependence on Nvidia due to the competition from Google, Microsoft, and AMD in providing alternative products.

Altman highlighted a positive relationship with major shareholder Microsoft and showed expectations for increased investments from the company.