报告时间：2024年10月9日（周三）

10:00-11:00

报告地点：中国人民大学明德主楼1016

报告嘉宾：Zenan Ling

报告主题：

Advancements in Understanding Deep Equilibrium Models: Bridging Theory and Practice

报告摘要

Advancements in Understanding Deep Equilibrium Models: Bridging Theory and Practice

A deep equilibrium model (DEQ) is implicitly defined through an equilibrium point of an infinite-depth weight-tied model with an input-injection. Instead of infinite computations, it solves an equilibrium point directly with root-finding and computes gradients with implicit differentiation. As a typical implicit neural network (NN), DEQ has recently emerged as a new neural network design paradigm, demonstrating remarkable success on various tasks. Nevertheless, the theoretical understanding of DEQs is still limited. In this talk, we will introduce several recent advancements in the theoretical comprehension of over-parameterized DEQs: (1) a novel non-asymptotic framework to establish the global convergence of the gradient descent (GD) associated with an over-parameterized DEQ; (2) a novel asymptotic framework for establishing the equivalence between implicit DEQs and explicit NNs in high dimensions. These findings leverage recent advances in high-dimensional analysis and random matrix theory.

作者简介

Zenan Ling is currently an assistant professor at Huazhong University of Science and Technology, School of Electronic Information and Communications. Prior to this, he was a postdoctoral researcher at Peking University, School of Intelligence Science and Technology, under the supervision of Prof. Zhouchen Lin. He received his Ph.D. in 2020 from Shanghai Jiao Tong University, where he worked under the supervision of Prof. Robert C. Qiu. He earned his B.S. degree in Mathematics from Nanjing University in 2015. His research interests broadly include machine learning, signal processing, random matrix theory, and high-dimensional statistics. His research has been funded by the National Natural Science Foundation of China and the Natural Science Foundation of Hubei Province.