Chapter 55: Environmental Impact & Green AI

Chapter opener illustration: Environmental Impact & Green AI.

"The carbon cost of training is a one-time bill; the carbon cost of inference compounds with every user."
Compass, Carbon-Aware AI Agent

Looking Back

Chapter 54 covered transparency; this chapter covers a different kind of disclosure: environmental impact. We work through training-time vs inference-time energy, regional grid carbon intensity, water use, and the small efficiency choices that compound across millions of inferences.

Chapter Overview

Training a frontier LLM emits tons of CO2 equivalent, but inference at scale emits more. This chapter covers the energy and carbon footprint of LLM systems, the measurement frameworks that quantify it, and the engineering choices (model size, hardware, batching, regional energy mix) that reduce environmental impact without trading away product quality.

Big Picture

The environmental accounting for LLMs, in three moves: measure it, cut it, and comply. Section 55.1 quantifies training and inference footprints with first-principles formulas and efficiency metrics (FLOPs/token, MFU, tokens/kWh). Section 55.2 reduces the footprint through architectural levers (MoE, distillation, quantization), region selection, and carbon-tracking tooling, while watching for the rebound effect. Section 55.3 operates under compliance: energy profiling, green-inference strategies, EU AI Act Article 53 environmental disclosure, and a production Green-AI checklist.

Note: Learning Objectives

Account for training and inference energy, water, and carbon footprint at the workload level.
Apply Green-AI techniques (model size choice, quantization, batching, regional energy mix) to a target deployment.
Compare hardware lifecycle impacts across NVIDIA, AMD, and emerging accelerator silicon.
Architect a carbon-accounting workflow for an enterprise LLM program.
Evaluate the trade-off between model quality and environmental impact for a specific product.

Prerequisites

Inference optimization from Chapter 9
Pretraining and scaling laws from Chapter 6
Compute-planning material covered later in the book provides additional context

Sections

What's Next?

This chapter begins with Section 55.1: Quantifying the Environmental Cost and builds through mitigation (55.2) to compliance and operations (55.3). Each section builds on the previous one, so we recommend reading them in order.

Further Reading

Foundational Papers

Strubell, E., Ganesh, A., & McCallum, A. (2019). "Energy and Policy Considerations for Deep Learning in NLP." ACL. arXiv:1906.02243. The paper that put NLP carbon costs on the field's agenda; first systematic accounting of training emissions for transformer-class models.

Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., et al. (2021). "Carbon Emissions and Large Neural Network Training." arXiv preprint. arXiv:2104.10350. Google's full accounting of GPT-3, T5, Meena, and GShard emissions; the source for the 4M factors (Model, Machine, Mechanization, Map) framework.

Luccioni, A. S., Viguier, S., & Ligozat, A.-L. (2023). "Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model." JMLR, 24(253). arXiv:2211.02001. Cradle-to-grave LCA of an open foundation model including hardware embodied emissions and idle power, the methodological reference for Section 55.1.

Water and Inference Cost

Li, P., Yang, J., Islam, M. A., & Ren, S. (2023). "Making AI Less Thirsty: Uncovering and Addressing the Secret Water Footprint of AI Models." arXiv preprint. arXiv:2304.03271. The first detailed accounting of on-site and off-site water consumption for LLM training and inference; the basis for the data-center water-budget discussion.

Luccioni, A. S., Jernite, Y., & Strubell, E. (2024). "Power Hungry Processing: Watts Driving the Cost of AI Deployment?" FAccT. arXiv:2311.16863. Benchmarks per-query inference energy across modalities; complements training-focused work with the deployment-time numbers that dominate at scale.