We hear many concerns about the new wave of generative AI, from the human jobs it could replace to its potential for creating misinformation and hacks.
But one aspect that is often overlooked is the enormous amount of energy these systems consume. In the not-so-distant future, the technology could consume as much electricity as an entire country.
Alex de Vries, a researcher at the Free University of Amsterdam, is the author of the paper “The Growing Energy Footprint of Artificial Intelligence,” which examines the environmental impact of AI systems.
Phases and usage of AI since its beginnings
De Vries points out that the training phase of large language models typically consumes the most energy, which has been the focus of sustainability research in AI.
After training, the models are deployed in a production environment and enter the inference phase. In the case of ChatGPT, this involves generating live responses to user queries.
There has been little research on the inference phase, but De Vries believes there are indications that this period can significantly contribute to the life cycle costs of an AI model.
According to the research firm SemiAnalysis, OpenAI needed 3,617 Nvidia HGX A100 servers, totaling 28,936 GPUs, to support ChatGPT, resulting in a daily energy demand of 564 MWh.
For comparison, it is estimated that during the training phase of GPT-3, 1,287 MWh were used, making the energy demand during the inference phase considerably higher.
Google, which reported that 60% of its AI-related energy consumption from 2019 to 2021 came from inference, is integrating AI capabilities into its search engine.
As early as February, Alphabet’s President, John Hennessy, stated that a single user interaction with an AI-powered search service “probably costs ten times more than a standard keyword search.”
With an energy demand of 6.5 kW per server, this would translate to a daily electricity consumption of 80 GWh and an annual consumption of 29.2 TWh, approximately the same amount used by Ireland each year.
It is unlikely to happen soon, among other reasons because Nvidia does not have the production capacity to supply more than half a million HGX servers, and it would cost Google $100 billion to purchase them.
