https://openreview.net/forum?id=650J0YFjnV
PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training | OpenReview
Training Deep Neural Networks (DNNs) with billions of parameters generally involves pipeline-parallel (PP) execution. Unfortunately, PP model training can use...
using gpusllm trainingbubbles