H100 Hopper Accelerator Now in Full Manufacturing, DGX Delivery In Q1’23

With NVIDIA’s fall GTC occasion in full swing, the corporate touched upon the majority of its core enterprise in a technique or one other on this morning’s keynote. On the enterprise aspect of issues, one of many longest-awaited updates was the cargo standing of NVIDIA’s H100 “Hopper” accelerator, which at introduction was slated to land in Q3 of this yr. Because it seems, with Q3 already practically over H100 isn’t going to make its Q3 availability date. However, in response to NVIDIA the accelerator is in full manufacturing, and the primary programs can be delivery from OEMs in October.

First revealed again in March at NVIDIA’s annual spring GTC occasion, the H100 is NVIDIA’s next-generation excessive efficiency accelerator for servers, hyperscalers, and related markets. Primarily based on the Hopper structure and constructed on TSMC’s 4nm “4N” course of, H100 is the follow-up to NVIDIA’s very profitable A100 accelerator. Amongst different adjustments, the latest accelerator from the corporate implements HBM3 reminiscence, assist for transformer fashions inside its tensor cores, assist for dynamic programming, an up to date model of multi-instance GPU with extra sturdy isolation, and a complete lot extra computational throughput for each vector and tensor datatypes. Primarily based round NVIDIA’s hefty 80 billion transistor GH100 GPU, the H100 accelerator can also be pushing the envelope when it comes to energy consumption, with a most TDP of 700 Watts.

Provided that NVIDIA’s spring GTC occasion didn’t exactly align with their manufacturing window for this technology, the H100 announcement earlier this yr said that NVIDIA can be delivery the primary H100 programs in Q3. Nonetheless, NVIDIA’s up to date supply objectives outlined right now imply that the Q3 date has slipped. The excellent news is that H100 is in “full manufacturing”, as NVIDIA phrases it. The unhealthy information is that it will appear that manufacturing and integration didn’t begin fairly on time; at this level the corporate doesn’t count on the primary manufacturing programs to succeed in prospects till October, the beginning of This fall.

Throwing an additional spanner into issues, the order through which programs and merchandise are rolling out is actually being reversed from NVIDIA’s regular technique. Reasonably than beginning with programs primarily based on their highest-performance SXM kind issue components first, NVIDIA’s companions are as an alternative beginning with the decrease performing PCIe playing cards. That’s to say that the primary programs delivery in October can be utilizing the PCIe playing cards, and it’ll solely be later within the yr that NVIDIA’s companions ship programs that combine the sooner SXM playing cards and their HGX service board.

In the meantime, NVIDIA’s flagship DGX programs, that are sometimes among the many very first programs to ship, at the moment are going to be among the many final. NVIDIA is opening pre-orders for DGX H100 programs right now, with supply slated for Q1 of 2023 – 4 to 7 months from now. That is excellent news for NVIDIA’s server companions, who within the final couple of generations have needed to wait to go after NVIDIA, however it additionally signifies that H100 as a product won’t be able to place its finest foot ahead when it begins delivery in programs subsequent month.

In a pre-briefing with the press, NVIDIA didn’t provide an in depth clarification as to why H100 has ended up delayed. Although talking at a excessive degree, firm representatives did state that the delay was not for part causes. In the meantime, the corporate cited the relative simplicity of the PCIe playing cards given that PCIe programs are delivery first; these are largely plug-and-play inside generic PCIe infrastructure, whereas the H100 HGX/SXM programs had been extra complicated and took longer to complete.

There are some notable function variations between the 2 kind components, as properly. The SXM model is the one one which makes use of HBM3 reminiscence (PCIe makes use of HBM2e), and the PCIe model requires fewer working SMs (114 vs. 132). So there’s some wiggle room right here for NVIDIA to cover early yield points, if certainly that is even an element.

Complicating issues for NVIDIA, the DGX H100 programs are primarily based on Intel’s repeatedly delayed 4^th technology Xeon Scalable processors (Sapphire Rapids), which in the intervening time nonetheless should not have a launch knowledge utterly nailed down. Much less optimistic projections have It launching in Q1, which does align with NVIDIA’s personal launch date – although this may occasionally very properly simply be coincidence. Both method, the dearth of normal availability for Sapphire Rapids isn’t doing NVIDIA any favors right here.

Finally, with NVIDIA unable to ship DGX till subsequent yr, it’s going to be NVIDIA’s server companions which might be first out the door with HGX programs – presumably utilizing current-generation hosts, or presumably AMD’s Genoa platform if it’s prepared in time. Among the many companies slated to ship H100 programs are the standard suspects, together with Supermicro, Dell, HPE, Gigabyte, Fujitsu, Cisco, and Atos.

In the meantime, for purchasers who’re desirous to check out H100 earlier than they purchase any {hardware}, H100 is now accessible on NVIDIA’s LaunchPad service.

Lastly, whereas we’re with regards to H100, NVIDIA can also be utilizing this week’s GTC to announce an replace to licensing for his or her NVIDIA AI Enterprise software program stack. H100 now comes with a 5-year license for the software program, which is notable since a 5 yr subscription is often $8000 per CPU socket.