https://www.deepspeed.ai/tutorials/megatron/
Megatron-LM GPT2 - DeepSpeed
If you haven’t already, we advise you to first read through the Getting Started guide before stepping through this tutorial.
megatronlmgpt2deepspeed
https://huggingface.co/hf-internal-testing/tiny-random-gpt2/tree/937b4d23b6648f5a1a0d1247b939b26981798903
hf-internal-testing/tiny-random-gpt2 at 937b4d23b6648f5a1a0d1247b939b26981798903
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
internal testinghftinyrandomgpt2
https://resources.wolframcloud.com/NeuralNetRepository/resources/GPT2-Transformer-Trained-on-WebText-Data/
GPT2 Transformer - Wolfram Neural Net Repository
Generate text in English and represent text as a sequence of vectors
neural net repositorygpt2transformerwolfram