A brand new white paper from Google particulars the corporate’s use of optical circuit switches in its machine studying coaching supercomputer, saying that the TPU v4 mannequin with these switches in place gives improved efficiency and extra vitality effectivity than general-use processors.
Google’s Tensor Processing Items — the essential constructing blocks of the corporate’s AI supercomputing methods — are basically ASICs, that means that their performance is inbuilt on the {hardware} stage, versus the overall use CPUs and GPUs utilized in many AI coaching methods. The white paper particulars how, by interconnecting greater than 4,000 TPUs by means of optical circuit switching, Google has been capable of obtain speeds 10 instances quicker than earlier fashions whereas consuming lower than half as a lot vitality.
Aiming for AI efficiency, value breakthroughs
The important thing, in keeping with the white paper, is in the way in which optical circuit switching (carried out right here by switches of Google’s personal design) allows dynamic modifications to interconnect topology of the system. In comparison with a system like Infiniband, which is often utilized in different HPC areas, Google says that its system is cheaper, quicker and significantly extra vitality environment friendly.
“Two main architectural options of TPU v4 have small price however outsized benefits,” the paper stated. “The SparseCore [data flow processors] accelerates embeddings of [deep learning] fashions by 5x-7x by offering a dataflow sea-of-cores structure that enables embeddings to be positioned wherever within the 128 TiB bodily reminiscence of the TPU v4 supercomputer.”
In keeping with Peter Rutten, analysis vp at IDC, the efficiencies described in Google’s paper are largely as a result of inherent traits of the {hardware} getting used — well-designed ASICs are virtually by definition higher suited to their particular job than basic use processors attempting to do the identical factor.
“ASICs are very performant and vitality environment friendly,” he stated. “Should you hook them as much as optical circuit switches the place you possibly can dynamically configure the community topology, you have got a really quick system.”
Whereas the system described within the white paper is just for Google’s inside use at this level, Rutten famous that the teachings of the expertise concerned might have broad applicability for machine studying coaching.
“I’d say it has implications within the sense that it gives them a form of finest practices situation,” he stated. “It’s an alternative choice to GPUs, so in that sense it’s positively an fascinating piece of labor.”
Google-Nvidia comparability is unclear
Whereas Google additionally in contrast TPU v4’s efficiency to methods utilizing Nvidia’s A100 GPUs, that are widespread HPC parts, Rutten famous that Nvidia has since launched a lot quicker H100 processors, which can shrink any efficiency distinction between the methods.
“They’re evaluating it to an older-gen GPU,” he stated. “However in the long run it doesn’t actually matter, as a result of it’s Google’s inside course of for creating AI fashions, and it really works for them.”
Copyright © 2023 IDG Communications, Inc.
