https://aclanthology.org/2024.acl-short.10/
UltraSparseBERT: 99% Conditionally Sparse Language Modelling - ACL Anthology
Peter Belcak, Roger Wattenhofer. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2024.
language modellingsparseaclanthology