What's the environmental cost of AI?

With the growing application of artificial intelligence across all industries, which is unlocking new opportunities and benefits for several organizations, there is also growing skepticism around the environmental footprint of AI.

To explore the environmental cost of inference, CO2 AI was invited to participate in a panel, jointly hosted by the Organisation for Economic Co-operation and Development (OECD) and Institute of Electrical and Electronics Engineers (IEEE), on the sidelines of the AI Action Summit in Paris. Here are my key takeaways from the panel, which saw participation from stakeholders from UNIDO, LinkedIn, Google and more.

What gets measured, gets done

At CO2 AI, our ethos has always been to measure, measure, measure. We can only tackle a problem once we have accurate data, which is especially true in the case of assessing the environmental impact of an activity.

The overarching takeaway from the panel was the need to develop robust methodologies for assessing environmental impact of Large Language Models (LLMs). To drive more responsible decision-making, we need new metrics that account for the full lifecycle environmental impact of AI systems. This includes not only carbon emissions but also water usage and atmospheric pollution.

Efforts like those driven by Hugging Face and Cohere, to jointly develop an AI Energy Score rating that offers a clear and standardized benchmark for measuring AI energy consumption, ensures that the AI community can make informed, sustainable choices.

System-level optimization can help reduce energy usage

The panel discussions highlighted that while AI unlocks scalability and efficiency opportunities, its usage needs to be optimized to minimize cost and impact.

This means considering the broader architecture surrounding the model, including hardware choices and energy sources. For example, running models on energy-efficient chips, deploying inference at the edge to reduce energy-intensive data center workloads, and improving the power grids that support AI infrastructure can significantly reduce environmental impact.

At CO2 AI, sustainability is at the heart of the decisions we make. We start by setting the example and optimizing our own solution, in line with our mission of reducing 500 mT of carbon per year.

How we optimize our AI usage at CO2 AI

Companies have traditionally computed emissions through broad manual categorization, which is imprecise and error-prone. Our AI solution was developed to increase the precision and accuracy of emissions measurement for companies. It combines LLMs and embedding models in a Retrieval Augmented Generation (RAG) architecture. We have optimized the data pipelines to create the smallest and most relevant context window to optimize the use of AI, without retraining or fine-tuning.

Using cost as a relevant proxy for energy consumption, we can proudly claim that it is at a reasonable €100 for 1 million activities matched. This significantly reduces the environmental impact of measuring emissions and also brings down cost, compared to the alternative of dedicating six or more months of a full-time employee or consultant to the same task. By leveraging efficient AI inference, organizations can achieve accurate emissions tracking at a fraction of the cost and resource consumption.

Driving real impact with customers - AI for climate action

In addition - and most importantly - by adding precision, the solution unlocks new ways for them to reduce their emissions, for instance switching from one supplier to another for a certain material.

For instance, Reckitt, a consumer goods leader, moved from 333 representative products to detailed emission data for each of its 25,000 products in just under 4 months. They now use precise carbon data to dive into impact at ingredient, process, and packaging-level and have designed 25 reduction initiatives to carry with suppliers, securing the journey toward 50% emission reduction by 2030.

It’s an uphill battle but shying away is not an option

The environmental impact of AI has been a thorny point of discussion across social media and other platforms.

On one hand, as an AI-first company, we recognize first-hand the huge potential of this technology and are seeing the decarbonization benefits of it for our customers every day. On the other hand, as a company guided by the principles of sustainability, we’re aiming to better assess the environmental impact of our AI-usage and operations.

As AI usage grows, especially for inference, companies need to measure it accurately as part of emissions reporting. For this, we must create incentives for AI labs to provide accurate data through life-cycle analysis. It’s especially difficult as models are rapidly evolving and their sole focus is on developing new models that perform better than previous ones. It also requires creating standards to enable customers to compare apples to apples, by setting the same boundaries, e.g. including training or not.

This work is not new and is being carried out in various industries already, or instance to compute emissions from chemicals or automotive. Usually these are industry coalitions, which are led by companies themselves to create standards for measurement, such as Together for Sustainability in the chemicals industry or Catena-X in automotive.

We expect companies to push AI labs for this in the coming years, as inference is expected to grow exponentially. Profit and planet can go hand-in-hand and we have demonstrated that with the intelligent and energy-conscious deployment of AI in our product. We will continue to develop our advocacy toward customers to increase awareness of and education on the environmental cost of using AI.