报告时间：2022年3月9日

               上午9:00-10:00

报告地点：腾讯会议

（会议ID：478 326 680）

报告嘉宾：Linglong Kong

报告主题：Statistics and Optimization in Reinforcement Learning

报告摘要

Statistics and Optimization in Reinforcement Learning

Reinforcement Learning (RL) is a mathematical framework to develop intelligent agents that can learn the optimal behaviour that maximizes the cumulative reward by interacting with the environment. There are numerous successful applications in many fields. Statistics and optimization are becoming important tools for RL. In this talk, we will look at two of our recent developments. In the first example, we employ distributional RL for efficient exploration. In distributional RL, the estimated distribution of value function models both the parametric and intrinsic uncertainties. We propose a novel and efficient exploration method for deep RL that has two components: a decaying schedule to suppress the intrinsic uncertainty and an exploration bonus calculated from the upper quantiles of the learned distribution. In the second example, we study damped Anderson mixing for deep RL. Anderson mixing has been heuristically applied to RL algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Motivated by that, we provide a rigorous mathematical justification for the benefits of Anderson mixing in RL. Our main results establish a connection between Anderson mixing and quasi-Newton methods, prove that Anderson mixing increases the convergence radius of policy iteration schemes by an extra contraction factor, and propose a stabilization strategy. Besides the two examples, we will discuss some current progress and future directions on statistics and optimization in RL.

个人简介

Dr. Linglong Kong is an associate professor at the department of Mathematical and Statistical Sciences of the University of Alberta. He is a Canada Research Chair in Statistical Learning. He has published about 60 peer-reviewed manuscripts including top journals AOS, JASA and JRSSB, and top conferences NeurIPS, ICML, ICDM, AAAI and IJCAI. Currently, Linglong is serving as associate editors of Journal of the American Statistical Association and Canadian Journal of Statistics, guest associate editor of the Frontiers of Neurosciences, member of the Board of Directors of the Statistics Society of Canada and Western North American Region of the International Biometric Society, and the ASA Statistical Computing Session program chair. He served as a guest editor of Canadian Journal of Statistics, associate editor of  International Journal of Imaging Systems and Technology, and the ASA Statistical Imaging Session program chair. His research interests include  high-dimensional data analysis, neuroimaging data analysis, statistical machine learning, robust statistics and quantile regression, AI for smart health.