https://openreview.net/forum?id=hFPRLiUORV
DARTFormer: Finding The Best Type Of Attention | OpenReview
NAS can be used to identify the best efficient Transformer attention mechanism for a given task. This leads to the idea of using mixed attention in a...
finding the besttypeattentionopenreview