报告时间：2023年4月27日

               上午9:00-10:00

报告地点：腾讯会议

             （会议ID：704-939-880）

报告嘉宾：Renjie Liao

报告主题：Gaussian-Bernoulli RBMs Without Tears

报告摘要

Gaussian-Bernoulli RBMs Without Tears

We revisit the challenging problem of training Gaussian-Bernoulli restricted Boltzmann machines(GRBMs), introducing two innovations. We propose a novel Gibbs-Langeyin sampling algorithm that outperforms existing methods like Gibbs sampling. We modified the contrastive divergence(CD) algorithm in two ways:1) adding variance-dependent initial step sizes for negative sampling; 2) drawing initial negative samples from Gaussian noise. We show this modified CD along with gradient clipping is enough to robustly train GRBMs with large learning rates, thus removing the need for various tricks in the literature. Moreover, it enables GRBMs to generate samples starting from noise, thus allowing direct comparisons with deep generative models and improving evaluation protocols in the RBM literature. Experiments on Gaussian Mixtures, MNIST, FashionMNIST, and CelebA show GRBMs can generate good samples, despite their single-hidden-layer architecture. Our code is released: https://github.com/lrjconan/GRBM.

个人简介

Renjie Liao has been an Assistant Professor at the Department of Electrical and Computer Engineering (ECE) at the University of British Columbia (UBC) since January 2022. Before joining UBC, he was a Visiting Faculty Researcher at Google Brain, working with Geoffrey Hinton and David Fleet. He received his Ph.D. degree in CS in 2021 from the University of Toronto, under the supervision of Richard Zemel and Raquel Urtasun. During his Ph.D., he worked as a Senior Research Scientist at Uber Advanced Technologies Group. He received an M.Phil. degree in CS in 2015 from the Chinese University of Hong Kong and a B.Eng. degree in Automation in 2011 from Beihang University. He is broadly interested in machine learning and its intersection with computer vision, self-driving, healthcare, and other areas, with a focus on probabilistic and geometric deep learning.