Processing the Carbon Footprint of Large Language Models — A short study

Modern Natural Language Processing uses deep learning models extensively. Due to significant environmental impact of deep learning, cost benefit analysis including carbon footprint as well as accuracy measure has been suggested to better document the use of NLP methods for research deployment.

Deep Learning Programs can have a high environmental impact in terms of Green house gas emissions due in particular to the energy consumption of the computational facilities used to run them.

In a recent medical imagining study, it was suggested that increase of large language model carbon footprint does not translate into proportional accuracy gains. Measuring this impact is a first step for raising awareness and controlling the impact of NLP experiments.

The contributions of this study are Identifying tools available for measuring the environmental impact of NLP Experiments. This article and the subsequent ones will discuss about various factors and tools which can be used to determine the carbon footprints and environmental impacts of large language models.

A French group (EcoInfo) working on AI sustainability came up with different tools that are used for tracking and analyzing the energy consumption and carbon footprint of software applications and systems.

1. “Experiment Impact Tracker” is a tool that allows users to measure the energy consumption of software experiments or tests. It is a Python library that measures the energy consumption of a given experiment, analyzes the results, and presents them in a format that is easy to understand. It is particularly useful for researchers who want to understand the energy implications of different software configurations or algorithms.
2. “PyJoules” is another Python library that can be used to measure the energy consumption of software applications. It uses the hardware performance counters in modern CPUs to measure the energy consumption of a program at runtime. PyJoules can be used to identify energy bottlenecks in software applications and to optimize them for better energy efficiency.
3. “Carbon Tracker” is a tool that calculates the carbon footprint of software applications or systems. It takes into account the energy consumption of the hardware that is used to run the software, as well as the energy mix of the electricity that is used to power the hardware. By analyzing the carbon footprint of a software system, developers can identify areas where they can reduce energy consumption and improve the sustainability of their software applications.

Although these tools are quite not much useful for deep learning algorithms like the ones we will look upon in subsequent blogs but they became the building blocks to measuring the carbon emissions by software applications and systems.

References :

1. Bannour, N., Ghannay, S., Névéol, A., & Ligozat, A.-L. (2021). Evaluating the carbon footprint of NLP methods: A survey and analysis of existing tools. arXiv preprint arXiv:2106.11097.
2. Li, W., Wu, J., Gao, F., Shi, Y., & Liu, J. (2021). Carbon Footprint of Selecting and Training Deep Learning Models for Medical Image Analysis. Frontiers in Artificial Intelligence, 4, 54.
3. https://ecoinfo.cnrs.fr/