When does memorization hurt robust generalization?

主讲人：彭镜夫 

彭镜夫，清华大学丘成桐数学科学中心助理教授。主要从事统计学与机器学习理论研究，研究方向包括模型选择与模型平均、对抗机器学习及深度学习理论等。相关成果发表于Journal of the American Statistical Association、Journal of Econometrics 等国际顶尖学术期刊。

1  报告信息

时间 

2026年3月20日（周五）

14:00

地点 

中国人民大学通州校区经济学部楼253

2  报告摘要

Understanding the intrinsic relationship between memorization and generalization is a fundamental topic in modern machine learning theory. In this talk, I will present our recent research on how memorization/interpolation of training data influences the robust generalization of regression estimators under adversarial attacks.

When memorization is measured by the    training error, we establish the full spectrum of minimax rates for the adversarial risk of estimators at different memorization levels. Our results show that a high level of memorization substantially damages the minimax adversarial rates. When memorization is measured by the    training error, we prove that once the normalized squared   training error falls below the Bayes risk, no learning procedure can remain consistent even under very subtle input perturbations. In contrast, there exist methods whose    training error stays at or above the Bayes risk that can achieve consistency against all adversarial attacks with vanishing magnitude. 

These theoretical insights demonstrate that excessive memorization fundamentally degrades a model's performance under adversarial shifts in future inputs. Numerical experiments on both simulation and real datasets further support our theoretical findings.