The design can work with a large neural network more efficiently than GPU banks connected together. But the production and operation of the chip is a complex task that requires new methods of etching silicon elements, a design that involves downsizing, and a new water supply system to cool the giant chip.
To create a cluster of WSE-2 chips capable of running record-size AI models, Cerebras had to solve another engineering challenge: how to efficiently retrieve and output data from a chip. Conventional chips have their own memory, but Cerebras has developed a memory box called MemoryX. The company has also created software that allows you to partially store a neural network in this off-chip memory, and only computations are transferred to the silicon chip. And she created a hardware and software system called SwarmX that connects everything together.
“They can improve the scale of learning to a large extent, in addition to what anyone is doing today,” says Mike Demler, senior analyst at Linley Group and senior editor. Microprocessor report.
Demler says it is unclear what the market will be for the cluster, especially since some potential customers are already developing their own, more specialized chips. He adds that the actual performance of the chip in terms of speed, efficiency and cost is still unclear. Cerebras has not published any benchmark results.
“The new MemoryX and SwarmX technology has a lot of impressive engineering technology,” Demler says. “But just like the processor, it’s a highly specialized material; it only makes sense to train the biggest models ”.
Cerebras chips have so far been adopted by labs that need supercomputing power. Among the first clients are the Argonne National Laboratories, Lawrence Livermore’s National Laboratory, pharmaceutical companies including GlaxoSmithKline and AstraZeneca, and what Feldman describes as “military intelligence” organizations.
This shows that the Cerebras chip can be used not only to power neural networks; the calculations performed in these laboratories involve similar massive parallel mathematical operations. “And they always crave more computing power,” says Demler, who adds that the chip could become important for future supercomputers.
David Canter, an analyst with Real World Technologies and CEO of MLCommons, which measures the performance of various artificial intelligence algorithms and equipment, says he sees a future market for much larger artificial intelligence models. “I tend to believe in data-driven ML [machine learning], so we need large datasets that allow us to build larger models with more parameters, ”says Kanter.