A team at Stanford University, Palo Alto, Calif., including computer scientist Mary Wootters and electrical engineer H.-S. Philip Wong has designed a system that can run AI tasks faster and with less energy by harnessing eight hybrid chips, each with its own data processor built right next to its own memory storage.
Smart watches and other battery-powered electronics would be even smarter if they could run AI algorithms. But efforts to build AI-capable chips for mobile devices have so far hit a wall. In traditional electronics, separate chips process and store data, wasting energy as they toss data back and forth over a so-called “memory wall” that separates data processing and memory chips but must work together to meet the massive and continually growing computational demands imposed by AI. The team’s new algorithms combine several energy-efficient hybrid chips to create the illusion of one mega–AI chip.
“Transactions between processors and memory can consume 95 percent of the energy needed to do machine learning and AI, and that severely limits battery life,” said computer scientist Subhasish Mitra, senior author of a new study published in Nature Electronics.
“If we could have built one massive, conventional chip with all the processing and memory needed, we’d have done so, but the amount of data it takes to solve AI problems makes that a dream,” Mitra said. “Instead, we trick the hybrids into thinking they’re one chip, which is why we call this the Illusion System.”
The work builds on the team’s development of a new memory technology, RRAM, that stores data even when power is switched off – like flash memory – only faster and more energy efficiently. Their latest design incorporates a critical new element: algorithms that meld its eight, separate hybrid chips into one energy-efficient AI-processing engine.
The Stanford-led team built and tested its prototype with help from collaborators at the French research institute CEA-Leti and at Nanyang Technological University in Singapore. The team’s eight-chip system is just the beginning. In simulations, the researchers showed how systems with 64 hybrid chips could run AI applications seven times faster than current processors, using one-seventh as much energy.
The team also developed new algorithms to recompile existing AI programs, written for today’s processors, to run on the new multi-chip systems. Collaborators from Facebook helped test AI programs that validated their efforts. Next steps include increasing the processing and memory capabilities of individual hybrid chips and demonstrating how to mass produce them cheaply.
The researchers developed Illusion as part of the Electronics Resurgence Initiative (ERI), a $1.5 billion program sponsored by the Defense Advanced Research Projects Agency. Wong believes Illusion Systems could be ready for marketability within three to five years.
For more information:
Stanford University
https://www.uah.edu/
CEA-Leti
https://www.leti-cea.com
Nanyang Technological University
https://www.ntu.edu.sg
Image – Hardware and software innovations give eight chips the illusion that they’re one mega-chip working together to run AI. Courtesy of Stocksy / Drea Sullivan.
