https://openreview.net/forum?id=47ahBl70xb
Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and...
Deep neural networks based on linear RNNs interleaved with position-wise MLPs are gaining traction as competitive approaches for sequence modeling. Examples of...
linear recurrences
https://openreview.net/forum?id=bd8kppxyB3
Revisiting Glorot Initialization for Long-Range Linear Recurrences | OpenReview
Proper initialization is critical for Recurrent Neural Networks (RNNs), particularly in long-range reasoning tasks, where repeated application of the same...
long rangelinear recurrencesrevisitinginitializationopenreview