https://openreview.net/forum?id=rcsZNV9A5j&referrer=%5Bthe%20profile%20of%20Jianguo%20Li%5D(%2Fprofile%3Fid%3D~Jianguo_Li2)
Flash Multi-Head Feed-Forward Network | OpenReview
We explore Multi-Head FFN (MH-FFN) as a replacement of FFN in the Transformer architecture, motivated by the structural similarity between single-head...
feed forward networkmulti headflashopenreview