AI's Carbon Footprint

Artificial intelligence (AI), especially deep learning, is making a big impact globally, leading to major advances in technology. Deep neural networks have revolutionized fields like computer vision and computational photography, enabling applications like visual object detection, human behavior detection, and visual search. These innovations are transforming industries from healthcare to transportation, demonstrating their significant impact on our daily lives.

In academic circles, the success of these technologies is evident in their performance. For example, on the ImageNet dataset, the error rate in image recognition is about 11.5%, showcasing the strength of these algorithms. Beyond their technical achievements, AI technologies are also driving economic growth by creating new industries and job opportunities.

However, the rapid advancement of AI comes with challenges. The high cost of training complex AI models could limit access to this technology, favoring large corporations and leaving smaller players, like startups and academic researchers, behind.

And then, there’s the environmental cost (Hao, 2019).

Carbon footprint assessment of AI training at UMass Amherst.

Researchers at UMass Amherst (Strubell et al., 2020) assessed the life cycle of training large AI models, finding that it can emit over 626,000 pounds of CO2 equivalent—nearly five times the lifetime emissions of an average American car.

Neil Thompson from MIT CSAIL warns that the environmental impact of AI is another growing concern (Thompson et al., 2021). The huge computational power needed for training AI models leads to high energy use and increased carbon emissions, contributing to global warming.

To quantify this alarming picture, MIT researchers found that training to achieve a 1% error rate would potentially cost more than US$100 quintillion (that’s not a typo, it’s $100 \times 10^{18}$ US dollars) and result in 100 quintillion pounds ($45 × 10^{18}$ metric tonnes) of carbon emissions. Thompson argues that when expenses continue to spiral out of control, researchers will have little choice to shift their focus to more efficient algorithms.

If AI continues to grow in complexity and resource demands, it could become an economic and environmental threat. Thompson’s research suggests that we might face extreme costs and significant carbon emissions for only slight improvements in AI performance.

Moreover, AI systems, despite their progress, struggle with more complex tasks. As these systems become more complicated, they are harder to understand and control, raising ethical concerns.

The future of AI will likely depend on developing more efficient and environmentally friendly algorithms. While there are challenges ahead, let’s hope that the potential benefits make it worth pursuing.

References

2021

Deep Learning’s Diminishing Returns: The Cost of Improvement is Becoming Unsustainable

Neil C. Thompson, Kristjan Greenewald, Keeheon Lee, and 1 more author

IEEE Spectrum, May 2021

Abs HTML

Deep learning is now being used to translate between languages, predict how proteins fold, analyze medical scans, and play games as complex as Go, to name just a few applications of a technique that is now becoming pervasive. Success in those and other realms has brought this machine-learning technique from obscurity in the early 2000s to dominance today. • Although deep learning’s rise to fame is relatively recent, its origins are not. In 1958, back when mainframe computers filled rooms and ran on vacuum tubes, knowledge of the interconnections between neurons in the brain inspired Frank Rosenblatt at Cornell to design the first artificial neural network, which he presciently described as a “pattern-recognizing device.” But Rosenblatt’s ambitions outpaced the capabilities of his era—and he knew it. Even his inaugural paper was forced to acknowledge the voracious appetite of neural networks for computational power, bemoaning that “as the number of connections in the network increases…the burden on a conventional digital computer soon becomes excessive.” • Fortunately for such artificial neural networks—later rechristened “deep learning” when they included extra layers of neurons—decades of Moore’s Law and other improvements in computer hardware yielded a roughly 10-million-fold increase in the number of computations that a computer could do in a second. So when researchers returned to deep learning in the late 2000s, they wielded tools equal to the challenge.

2020

Energy and Policy Considerations for Modern Deep Learning Research

Emma Strubell, Ananya Ganesh, and Andrew McCallum

Proceedings of the AAAI Conference on Artificial Intelligence, Apr 2020

Abs HTML PDF

The field of artificial intelligence has experienced a dramatic methodological shift towards large neural networks trained on plentiful data. This shift has been fueled by recent advances in hardware and techniques enabling remarkable levels of computation, resulting in impressive advances in AI across many applications. However, the massive computation required to obtain these exciting results is costly both financially, due to the price of specialized hardware and electricity or cloud compute time, and to the environment, as a result of non-renewable energy used to fuel modern tensor processing hardware. In a paper published this year at ACL, we brought this issue to the attention of NLP researchers by quantifying the approximate financial and environmental costs of training and tuning neural network models for NLP (Strubell, Ganesh, and McCallum 2019). In this extended abstract, we briefly summarize our findings in NLP, incorporating updated estimates and broader information from recent related publications, and provide actionable recommendations to reduce costs and improve equity in the machine learning and artificial intelligence community

2019

Training a single AI model can emit as much carbon as five cars in their lifetimes

Karen Hao

Apr 2019

Abs HTML

The artificial-intelligence industry is often compared to the oil industry: once mined and refined, data, like oil, can be a highly lucrative commodity. Now it seems the metaphor may extend even further. Like its fossil-fuel counterpart, the process of deep learning has an outsize environmental impact.

References

2021

2020

2019

Enjoy Reading This Article?