Cerebras Systems, Los Altos, Calif., a startup dedicated to accelerating artificial intelligence computing, announces that it has developed the largest chip ever built: a WSE chip with 71.6 square inches of area.
Optimized for AI work, the Cerebras wafer scale engine (WSE) is a single chip that contains more than 1.2 trillion transistors and 46,225 square millimeters (71.6 square inches) in area. The WSE is 56.7 times larger than the largest graphics processing unit, which measures 815 square millimeters (1.26 square inches) and 21.1 billion transistors. The WSE also contains 3000 times more high-speed, on-chip memory, and has 10,000 times more memory bandwidth.
With vastly more silicon area than the largest graphics processing unit, the WSE provides more compute cores, tightly coupled memory for efficient data access, and an extensive high bandwidth communication fabric for groups of cores to work together.
The WSE contains 400,000 Sparse Linear Algebra (SLA) cores. Each core is flexible, programmable, and optimized for the computations that underpin most neural networks. Programmability ensures the cores can run all algorithms in the constantly changing machine learning field.
The 400,000 cores on the WSE are connected via the Swarm communication fabric in a 2D mesh with 100 Pb/s of bandwidth. Swarm is a massive on-chip communication fabric that delivers breakthrough bandwidth and low latency at a fraction of the power draw of traditional techniques used to cluster graphics processing units. It is fully configurable; software configures all the cores on the WSE to support the precise communication required for training the user-specified model. For each neural network, Swarm provides a unique and optimized communication path.
