https://openreview.net/forum?id=HfjAumxjT2
MuLoCo: Muon is a practical inner optimizer for DiLoCo | OpenReview
DiLoCo is a powerful framework for training large language models (LLMs) under networking constraints with advantages for increasing parallelism and...
is amuonpracticalinneroptimizer