https://arxiv.org/abs/2311.08105?ref=blog.gensyn.ai
[2311.08105] DiLoCo: Distributed Low-Communication Training of Language Models
Abstract page for arXiv paper 2311.08105: DiLoCo: Distributed Low-Communication Training of Language Models
communication trainingdistributedlowlanguagemodels