https://openreview.net/forum?id=l2qgijevG0
You can remove GPT2's LayerNorm by fine-tuning | OpenReview
The LayerNorm (LN) layer in GPT-style transformer models has long been a hindrance to mechanistic interpretability. LN is a crucial component required to...
you canfine tuningremovegpt2layernorm
https://www.servethehome.com/sambanova-sn10-rdu-at-hot-chips-33/hc33-sambanova-sn10-rdu-layernorm-pipelined-in-spaced-and-fused/
HC33 SambaNova SN10 RDU LayerNorm Pipelined In Spaced And Fused - ServeTheHome
HC33 SambaNova SN10 RDU LayerNorm Pipelined In Spaced And Fused
sambanovardulayernorm
https://huggingface.co/papers/2305.02582
Paper page - On the Expressivity Role of LayerNorm in Transformers' Attention
Join the discussion on this paper page
on thepaperexpressivity
https://openreview.net/forum?id=JiL0sYCGeP&referrer=%5Bthe%20profile%20of%20Gopala%20Anumanchipalli%5D(%2Fprofile%3Fid%3D~Gopala_Anumanchipalli1)
LayerNorm vs RMSNorm: A Geometric Perspective and the Case Against Mean Subtraction | OpenReview
This paper presents a novel geometric interpretation of LayerNorm and explores how LayerNorm influences the norm and orientation of hidden vectors in the...