Responsible Generative AI
mining ( which is reported by Cambridge University to be 121.36 terawatt-hours ( TWh ) a year – about the same energy use as the entire country of Argentina 27 ). If care is not taken , this high growth in energy use could lead to significant carbon emissions , and potential energy shortages .
7.2 GENAI ENERGY ESTIMATE
Training the GPT-3 model like the one used by Open-AI ’ s ChatGPT GenAI system is reported to require 3.14 x 10^23 Floating Point Operations ( FLOPs ) – 314 Zetta FLOPs 28 . This is assumed to be 32-bit floating point operations . Newer AI models , such as GPT4.0 are expected to use at least an order of magnitude more .
The next component of the calculation is to determine the FLOP capacity of a reasonable choice of processor infrastructure . Clusters of high-performance Graphics Processing Units with AI acceleration capabilities are the preferred choice for many AI compute infrastructures . One leading GPU server is the NVIDIA DGX-H100 29 , which has eight H100 Tensor Core GPU chips , a server rating of approximately 8 Peta FLOPs / second ( 32-bit float ), and a power consumption of 10.2KW .
Finally , the full computational capability of a GPU cluster can ’ t be applied to the model building problem due to inefficiencies in parallel execution , memory conflicts , inter-chip network inefficiencies and many other factors . This reference 30 suggests 33 % utilization of the available compute cycles is reasonable .
So , dividing the 314 Zetta FLOPS by 3 ( because of the 33 % efficiency factor ) needed to compute the GPT-3 model by the 8 Peta FLOPs per second ( 32 bit ) supplied by a DGX-H100 GPU cluster with 8 GPUs , we see about 118 million system seconds , or approximately 3.7 system years of continuous computation are needed for each learning run of the GPT3 model .
Running a DGX H100 for 3.7 years at 10.2KW requires about 334000 kWh of electric power . Adding an additional 20 % for cooling and data center power overhead , the electrical consumption due to a DGX H100 is 400000kWh per model run . Electrical sources in the US average about 0.855 pounds of CO2 emissions per kWh , so each training run would produce 171 tons of CO2 .
27 https :// www . bbc . com / news / technology-56012952
28 https :// lambdalabs . com / blog / demystifying-gpt-3
29
https :// resources . nvidia . com / en-us-dgx-systems / ai-enterprise-dgx ? ncid = pa-srch-goog- 122967 # cid = dl32 _ pa-srch-goog _ en-us
30 https :// openai . com / research / ai-and-compute 32
March 2024