https://openreview.net/forum?id=AzFhwMa6X0
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training |...
OpenDiLoCo is an open-source implementation and replication of the Distributed Low-Communication (DiLoCo) training method for large language models. We provide...
open sourceframework