Couldn’t attend Transform 2022? Check out all the top sessions in our on-demand library now! Look here.
Can artificial intelligence (AI) analog hardware—rather than digital—use fast, energy-efficient processing to solve the rising costs and environmental footprint of machine learning?
Researchers say yes: Logan Wright and Tatsuhiro Onodera, research scientists at NTT Research and Cornell University, envision a future where machine learning (ML) will be performed with new physical hardware, such as those based on photonics or nanomechanics. These unconventional devices could be applied in both edge and server settings, they say.
Deep neural networks, which are at the heart of today’s AI efforts, depend on the intensive use of digital processors such as GPUs. But for years, concerns have been raised about the monetary and environmental costs of machine learning, which are increasingly limiting the scalability of deep learning models.
A 2019 paper for example, from the University of Massachusetts, Amherst, conducted a life cycle assessment for training several common large AI models. It found that the process can emit more than 626,000 pounds of carbon dioxide equivalent — nearly five times the lifetime emissions of the average American car, including the production of the car itself.
During a session with NTT Research at VentureBeat Transform’s Executive Summit on July 19, CEO Kazu Gomi said that machine learning doesn’t have to rely on digital circuitry, but can instead run on a physical neural network. This is a kind of artificial neural network in which physical analog hardware is used to mimic neurons as opposed to software-based approaches.
“One of the obvious benefits of using analog systems over digital ones is the power consumption of AI,” he said. “The consumption problem is real, so the question is what are new ways to make machine learning faster and more energy efficient?”
Analog AI: More like the brain?
From the early history of AI, people tried not to think about creating digital computers, Wright noted.
“They were trying to think about how we could mimic the brain, which of course isn’t digital,” he explained. “What I have in mind is an analog system, and it’s actually much more efficient at performing the kinds of computations that take place in deep neural networks than today’s digital logic circuits.”
The brain is an example of analog hardware for doing AI, but others include systems that use optics.
“My favorite example is waves, because a lot of things like optics are based on waves,” he said. “For example, in a bathtub, you could formulate the problem of coding a series of numbers. At the front of the bathtub you can set up a wave and the height of the wave will give you this vector X. You let the system evolve for a while and the wave propagates to the other end of the bathtub. After a while you can measure the height of that, and then you get a series of numbers again.”
In essence, nature itself can perform calculations. “And you don’t have to hook it up to anything,” he said.
Analog AI Hardware Approaches
Researchers across the industry are using different approaches to developing analog hardware. IBM Researchfor example, has invested in analog electronics, especially memristor technology, to perform machine learning calculations.
“It’s promising,” Onodera said. “These memristor circuits have the property that information is naturally calculated by nature as the electrons ‘flow’ through the circuit, potentially having much lower power consumption than digital electronics.”
However, NTT Research is focused on a more general framework that is not limited to memristor technology. “Our work is aimed at enabling other physical systems, for example systems based on light and mechanics (sound), to perform machine learning,” he said. “This allows us to create smart sensors in the native physical domain where the information is generated, such as in the case of a smart microphone or a smart camera.”
Startups, including Mythic, are also targeting analog AI using electronics — which Wright says is a “great move, and it’s probably the lowest-risk way to get into analog neural networks.” But it’s also incremental and has a limited ceiling, he added: “There’s only so much improvement in performance possible if the hardware is still electronics-based.”
Long-term potential of analog AI
Several startups, such as LightMatter, Lightelligence and Luminous Computing, use light instead of electronics for computing – also known as photonics. This is riskier, less mature technology, Wright said.
“But the long-term potential is much more exciting,” he said. “Light-based neural networks could be much more energy efficient.”
However, light and electrons aren’t the only things you can turn into a computer, especially for AI, he added. “You could get it from biological materials, electrochemistry (like our own brains), or from liquids, acoustic waves (sound) or mechanical objects, modernizing the earliest mechanical computers.”
MIT research, for example, announced last week that it had new protonic programmable resistors, a network of analog artificial neurons and synapses that can do calculations in much the same way as a digital neural network by repeatedly repeating series of programmable resistors in intricate layers. they used a “practically inorganic material in the manufacturing process,” they said, allowing their devices to “work 1 million times faster than previous versions, which is also about 1 million times faster than the synapses in the human brain.”
NTT Research says it is going a step further than all these approaches and asking much bigger, much longer term questions: What can we make a computer out of? And if we want to achieve AI systems with the highest speed and energy efficiency, what should we make them physically out of?
“Our paper provides the first answer to these questions by telling us how to create a neural network computer using any physical substrate,” says Logan. “And so far, our calculations suggest that making these weird computers will one day make a lot of sense, because they could be much more efficient than digital electronics, and even analog electronics. Light-based neural network computers seem to be the best approach so far.” , but even that question has not yet been fully answered.”
Analog AI not the only non-digital hardware bet
According to Sara Hooker, a former Google Brain researcher who currently heads the nonprofit Cohere research lab for AI, the AI industry is “in this really interesting hardware phase.”
Ten years ago, she explains, AI’s massive breakthrough was really a hardware breakthrough. “Deep neural networks only worked with GPUs, which were used for video games [and] were simply reused for deep neural networks,” she said.
The change, she added, was almost immediate. “Overnight, 13,000 CPUs required two GPUs overnight,” she said. “It was that dramatic.”
It’s very likely that there are other ways to represent the world that could be as powerful as digital, she said. “If even one of these data directions starts to show progress, it could unlock a lot of both efficiencies and different ways of learning representations,” she explains. “That makes it worthwhile for labs to support them.”
Hooker, whose essay for 2020 “The Hardware Lotteryexplored the reasons why various hardware tools have succeeded and failed, saying that the success of GPUs for deep neural networks “was actually a bizarre, happy coincidence — it was winning the lottery.”
GPUs, she explained, were never designed for machine learning — they were developed for video games. So much of the adoption of GPUs for AI use “depended on the right moment of reconciliation between the progress on the hardware side and the progress on the modeling side,” she said. “Making more hardware options available is the key ingredient because it allows for more unexpected moments when you see those breakthroughs.”
However, analog AI is not the only option that researchers are looking at when it comes to reducing the costs and CO2 emissions of AI. Researchers are betting in other areas, such as field-programmable gate arrays (FPGAs) as application-specific accelerators in data centers, which can reduce power consumption and increase operating speed. There are also efforts to improve software, she explained.
Analog, she said, “is one of the riskier bets.”
Expiration date on current approach
Still, there are risks to be taken, Hooker said. When asked if she thought the big tech companies support analog and other forms of alternative non-digital AI futures, she said: “One hundred percent. There is a clear motivation,” adding that the lack of sustainable public investment in a hardware landscape for the long term.
“It’s always been tricky when investments are solely based on companies because it’s so risky,” she said. “It often has to be part of a nationalist strategy to be a compelling long-term bet.”
Hooker said she wouldn’t bet on widespread adoption of analog AI hardware, but insists the research efforts are good for the ecosystem as a whole.
“It’s kind of like NASA’s first flight to the moon,” she said. “There are so many scientific breakthroughs that only happen by having a purpose.
And there is an expiration date for the industry’s current approach, she warned: “There is an understanding among people in the field that there has to be a bet on riskier projects.”
The future of analog AI
The NTT researchers made it clear that the earliest, narrowest applications of their analog AI work will take at least 5-10 years to mature — and even then, they will likely be used first for specific applications such as edge.
“I think in the short term most applications will be on the fringe, where there are fewer resources, where you may not have as much power,” Onodera says. “I think that’s really where the most potential is.”
One of the things the team is thinking about is which types of physical systems are the most scalable and offer the greatest advantage in terms of energy efficiency and speed. But as far as entering the deep learning infrastructure, it will likely happen incrementally, Wright said.
“I think it would come to market slowly, with a multi-layer network with maybe the front-end on the analog domain,” he said. “I think that’s a much more sustainable approach.”
The mission of VentureBeat is a digital city square for tech decision makers to gain knowledge about transformative business technology and transactions. Learn more about membership.