https://www.nvidia.com/en-us/on-demand/session/gtcspring22-s41904/
How to Avoid the Staggering Cost of Training State-of-the-art Large Language Models S41904 | GTC...
As state-of-the-art language models grow in capability and performance, so does their size, which now measures in billions or even trillions of parameters