https://openreview.net/forum?id=Gl4AsqInti&referrer=%5Bthe%20profile%20of%20Hossein%20Mobahi%5D(%2Fprofile%3Fid%3D~Hossein_Mobahi2)
Recent work has shown that first order methods like SAM which implicitly penalize second order information can improve generalization in deep learning....
hessianstructureexplainsmysteriessharpness
https://www.mdpi.com/2076-3417/12/19/9943
Federated Learning is a widely adopted method for training neural networks over distributed data. One main limitation is the performance degradation that...
it matterscomparisonsusinglayerwise
https://openreview.net/forum?id=8bpPxz2bJI&referrer=%5Bthe%20profile%20of%20Ziv%20Goldfeld%5D(%2Fprofile%3Fid%3D~Ziv_Goldfeld1)
The Gromov-Wasserstein (GW) distance, rooted in optimal transport (OT) theory, quantifies dissimilarity between metric measure spaces and provides a framework...
sample complexitygromovwassersteindistancesentropic
https://www.digitalocean.com/community/tutorials/regularization-in-machine-learning-lasso-ridge-elastic-net
Discover Ridge Regression in machine learning with Python examples. Learn the formula, how it works, and how it compares to Lasso and ElasticNet.
elastic netsimplifyingregularizationpartridge
https://arxiv.org/abs/2310.10810
Abstract page for arXiv paper 2310.10810: Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms
reinforcement learningrobustmultiagentvia
https://www.inference.vc/notes-on-the-origin-of-implicit-regularization-in-stochastic-gradient-descent/
I wanted to highlight an intriguing paper I presented at a journal club recently: * Samuel L Smith, Benoit Dherin, David Barrett, Soham De (2021) On the Origin...
on thenotesoriginimplicitregularization
https://arxiv.org/abs/2210.09188
Abstract page for arXiv paper 2210.09188: Sub-8-bit quantization for on-device speech recognition: a regularization-free approach
subbitquantizationdevicespeech
https://www.tensorflow.org/api_docs/python/tf/compat/v1/losses/get_regularization_losses?authuser=2
Gets the list of regularization losses.
tfcompatlossesgetregularization
https://deepai.org/publication/ensemble-model-with-batch-spectral-regularization-and-data-blending-for-cross-domain-few-shot-learning-with-unlabeled-data
06/08/20 - Deep learning models are difficult to obtain good performance when data is scarce and there are domain gaps. Cross-domain few-shot...
spectral regularizationdata blendingensemblemodelbatch