https://openreview.net/forum?id=pZCYG7gjkKz
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method | OpenReview
Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines...