How Google Tripled AI Chip Carbon Efficiency

Google has released a first-of-its-kind study on the full lifecycle emissions of its AI accelerator chips. 

“The study found that innovation in our chip hardware design led to a 3x improvement in the carbon-efficiency of AI workloads over two generations and that decarbonising our electricity-related emissions will drive the biggest carbon reductions for our AI footprint,” explains Kate Brandt, Chief Sustainability Officer at Google.

“At Google, we know AI can drive transformative innovation in areas like information, optimisation and prediction. 

“We also know it’s equally important to manage its environmental impacts, and we’re working to do that through efficient infrastructure, model optimisation and emissions reductions.

“This study is an important step in those efforts, unlocking critical insights for Google and others looking to reduce emissions across the full lifetime of AI hardware.”

Adam Elman, Director of Sustainability EMEA at Google, says: “This is just the beginning with huge opportunities to continue optimising hardware and software for carbon efficiency.”

A new metric: Compute Carbon Intensity

Compute Carbon Intensity (CCI) is a new metric developed for this study that Google believes can enable greater transparency and innovation across the industry.

The study examined five models of Tensor Processing Unit (TPU) hardware to estimate their full lifecycle emissions and understand how hardware design decisions impact carbon efficiency. 

TPUs are specialised hardware accelerators that help advance AI.

“Their efficiency impacts AI's environmental sustainability. This progress is due to more efficient hardware design, which means fewer carbon emissions for the same AI workload,” explains Robert Little, Sustainability Strategy Lead for gTech at Google.

CCI quantified an AI accelerator chip’s carbon emissions per unit of computation – this is measured in grams of CO₂ per Exa-FLOP. 

A lower CCI score therefore means lower emissions from the hardware platform for a given AI workload. 

Google has used CCI to track its progress in increasing the carbon efficiency of its TPUs. 

Key takeaways from Google’s study

Google’s research found an improvement of three times in the CCI of its TPU chips over four years. 

This is across the evolution from TPU v4 to Trillium. 

By choosing newer generations of TPUs, its customers generate fewer carbon emissions for the same AI workload. 

Operational electricity emissions make up more than 70% of a Google TPU’s lifetime emissions, which shows the importance of improving the energy efficiency of AI chips.

It is also vital to reduce the carbon intensity of the electricity powering these chips. 

While operational emissions make up most of a TPU’s lifetime emissions, manufacturing emissions still play a part and their share of total emissions will increase as operational emissions decrease. 

Google’s study’s detailed manufacturing LCA helps it to target its manufacturing decarbonisation efforts towards the highest-impact initiatives and it is actively working with its supply chain partners to reduce these emissions. 

“These findings highlight the importance of optimising both hardware and software for a sustainable AI future,” Robert says. 

“It's important to remember where AI has important implications for reducing emissions and fostering sustainability.”

Explore the latest edition of Sustainability Magazine and be part of the conversation at our global conference series, Sustainability LIVE. 

Discover all our upcoming events and secure your tickets today.

Sustainability Magazine is a BizClik brand