https://openreview.net/forum?id=f0YBPErlfC&referrer=%5Bthe%20profile%20of%20Bo-Wen%20Zhang%5D(%2Fprofile%3Fid%3D~Bo-Wen_Zhang1)
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies | OpenReview
In recent years, with the rapid application of large language models across various fields, the scale of these models has gradually increased, and the...