https://openreview.net/forum?id=r5nb8FTZkY&referrer=%5Bthe%20profile%20of%20Surbhi%20Goel%5D(%2Fprofile%3Fid%3D~Surbhi_Goel1)
Progressive distillation induces an implicit curriculum | OpenReview
Knowledge distillation leverages a teacher model to improve the training of a student model. A persistent challenge is that a better teacher does not always...
implicit curriculumprogressivedistillationinducesopenreview
https://openreview.net/forum?id=9pwlOZneBh&referrer=%5Bthe%20profile%20of%20Surbhi%20Goel%5D(%2Fprofile%3Fid%3D~Surbhi_Goel1)
Progressive distillation improves feature learning via implicit curriculum | OpenReview
Knowledge distillation, where a student model learns from a teacher model, is a widely-adopted approach to improve the training of small models. A known...
feature learningimplicit curriculumprogressivedistillationimproves