The Next Frontier of AI: Lower Emission Processing Using Analog Computers

Deep learning programs running on digital computers may undergo profound disruption in the coming years. Analog computers, previously the unwieldy behemoths covered in glass dials and criss-crossing wires, could be making a comeback as a compute and energy efficient option. 

Analog computers started to fall out of fashion from the 1970s onwards with the rise of general-purpose digital technology that allowed for flexible programming. However, now as AI algorithms push the physical limits of digital computers and increasingly intensive computations are required to produce results, there’s a buzz in the tech world about returning to analog.

In the current deep learning paradigm, increases in performance are largely dependent on computational power. In the vast majority of cases, the more expansive the training set is for a program to learn from, the more performant the program becomes. Additionally, as AI use-cases and processing tasks increase in complexity, underlying neural network models need to be configured with more nodes as well as layers, meaning more matrix calculations would be needed.

“For a linear gain in performance, an exponentially larger model is required, which can come in the form of increasing the amount of training data or the number of experiments, thus escalating computational costs, and therefore carbon emissions”[1]

The dramatic increase in processing power and data storage needed for the new generation of AI algorithms means a corresponding increase in energy consumption. Many researchers, notably Strubell et al. (2019) [2] and Henderson et al. (2020) [3], have been analyzing and reporting on the massive amounts of energy that AI applications use to both power and cool data centers, thereby generating CO2.

Methods to reduce the carbon footprint of AI include using compute-efficient machine learning that leverages compressed network architectures [4] as well as lessening input volumes with data minimization techniques. However, a more speculative and drastic paradigm shift is brewing below the surface of the AI landscape that has the potential to greatly improve efficiency: the analog computer.

AI Hardware Spotlight: Mythic Inc.’s Analog Chip

Disclaimer: As of this article’s publication, neither the author nor MAIEI has any affiliation with Mythic Inc. This article, written solely for the website, originates from author research and references publically available Mythic documents.

Current AI applications on digital machines are memory and compute-intensive. In order to facilitate the matrix multiplication operations underlying machine learning tasks, data must be moved from memory store to the compute engine and this limitation, known as the Von Neumann bottleneck, accounts for a significant amount of power being dissipated. Analog computers can be configured to perform in-memory computation which removes this bottleneck, offering more operations for less energy usage.

Mythic Inc. is a groundbreaking company developing and commercializing analog AI chips. With their M1076 Analog Matrix Processor, Mythic advertises up to 25 TOPS (trillions of operations per second) per 3 to 4 watts usage [5]. Representing a little over 6 TOPS per watt, this figure holds efficiencies over their competition when it comes to energy usage, as NVIDIA’s Jetson AGX Xavier advertises 32 TOPs per 10 watts or 3.2 TOPS per watt. [6]

Note that TOPS per watt metrics are not the end-all of measuring performance and has even been criticized as an imprecise count since firms may measure operations in varied ways and certain types of chips could specialize in different types of operations. Also keep in mind that comparing an analog chip to a digital chip may not be entirely appropriate. Analog chips are currently being explored to tackle more directed use-cases, like matrix multiplication tasks, as it lacks the flexibility and reprogrammable nature of a digital chip. Additionally, using a nondeterministic analog system also affects the explainability of its calculations with its higher sensitivity to physical influences.

Another drawback may include physical waste. If we can imagine a world with the mass adoption of analog systems in our applications, we can also imagine that it would be easier to dispose of analog machines than to extract its components or reprogram it for future use. 

However, as it stands today, the efficiency that can be gained and the magnitude of energy that can be saved from using analog systems is too tempting to not try.

References

[1] Dhar, P. The carbon impact of artificial intelligence. Nat Mach Intell 2, 423–425 (2020). https://doi.org/10.1038/s42256-020-0219-9

[2] Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243.

[3] Henderson, P., Hu, J., Romoff, J., Brunskill, E., Jurafsky, D., & Pineau, J. (2020). Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248), 1-43.

[4] Gupta, A., Lanteigne, C., & Kingsley, S. (2020). SECure: A social and environmental certificate for AI systems. arXiv preprint arXiv:2006.06217.

[5] https://www.prnewswire.com/news-releases/mythic-expands-product-lineup-with-new-scalable-power-efficient-analog-matrix-processor-for-edge-ai-applications-301306344.html

[6] https://developer.nvidia.com/embedded/jetson-agx-xavier