[Statlist] ETH Young Data Science Researcher Seminar Zurich, Virtual Seminar by Stefan Perko, University of Jena, 19.05.2022

[Kaiser-Heinzmann Susanne]

We are glad to announce the following talk in the virtual ETH Young Data Science Researcher Seminar Zurich

"Towards diffusion approximations for stochastic gradient descent without replacement“
by Stefan Perko, University of Jena

Time: Thursday, 19 May 2022, 15.00 - 16.00 CEST
Place: Zoom at https://ethz.zoom.us/j/62895316484

Abstract: Stochastic gradient descent without replacement or reshuffling (SGDo) is predominantly used to train machine learning models in practice. However, the mathematical theory of this algorithm remains underexplored compared to its "with replacement" and "infinite data" counterparts. We propose a stochastic, continuous-​time approximation to SGDo based on a family of stochastic differential equations driven by a stochastic process we call epoched Brownian motion, which encapsulates the behavior of reusing the same data points in subsequent epochs. We investigate this diffusion approximation by considering an application of SGDo to linear regression. Explicit convergence results are derived for constant learning rates and a sequence of learning rates satisfying the Robbins-​Monro conditions. Finally, the validity of continuous-​time dynamics are further substantiated by numerical experiments.


M. Azadkia, G. Chinot, J. Hörrmann, M. Löffler, A. Taeb, N. Zhivotovskiy


Seminar website: https://math.ethz.ch/sfs/news-and-events/young-data-science.html

Young Data Science Researcher Seminar Zurich – Seminar for Statistics | ETH Zurich
math.ethz.ch<https://math.ethz.ch/>






__________________
ETH Zürich
Seminar für Statistik
Sekretariat
Rämistrasse 101
CH-8092 Zürich









	[[alternative HTML version deleted]]