Deep Unsupervised Learning using Nonequilibrium Thermodynamics

The paper tackles a central problem in machine learning: modeling complex datasets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation remain analytically or computationally tractable. Inspired by non-equilibrium statistical physics, the method systematically and slowly destroys structure in a data distribution through an iterative forward diffusion process, then learns a reverse diffusion process that restores structure in the data.

The result is a generative model that achieves flexibility and tractability simultaneously. The approach allows rapid learning, sampling, and probability evaluation in deep generative models with thousands of layers or time steps, as well as computation of conditional and posterior probabilities under the learned model, and the authors release an open-source reference implementation of the algorithm.