机器学习与数据科学博士生系列论坛（第五十七期）—— On the Convergence Rates of Two-Time-Scale Stochastic Approximation
报告人：Yuze Han (PKU)
地点：腾讯会议 723 1564 5542
Two-time-scale stochastic approximation is a variant of the classic stochastic approximation (SA) to find the root of a system of two coupled equations. It has been widely used in various applications spanning stochastic optimization and reinforcement learning. In this algorithm, there are two iterates: the fast iterate and the slow iterate. The fast iterate is updated by using step sizes that are much larger than the ones used to update the slow iterate. Meanwhile, the update rule of the fast iterate depends on the slow iterate and vice versa. Despite the double dependence between the two time scales, decoupled convergence rates could also be established under certain conditions, e.g., the linear case.
In this talk, we first present some typical examples of two-time-scale SA and then discuss the asymptotic and non-asymptotic convergence rates. For the linear case, we show how to achieve decoupled convergence; for the nonlinear case, we compare the convergence results under different conditions.