Categorical Reparameterization with Gumbel-Softmax

Categorical variables are a natural way to represent discrete structure in the world, yet stochastic neural networks rarely use categorical latent variables because it is not possible to backpropagate through samples drawn from a categorical distribution. To overcome this, the paper presents an efficient gradient estimator that replaces the non-differentiable sample from a categorical distribution with a differentiable sample from a novel Gumbel-Softmax distribution, which has the essential property that it can be smoothly annealed into a categorical distribution.

The authors show that their Gumbel-Softmax estimator outperforms state-of-the-art gradient estimators on structured output prediction and unsupervised generative modeling tasks involving categorical latent variables, and that it enables large speedups on semi-supervised classification. By making discrete latent variables trainable end-to-end, the technique became a widely used building block for models that need differentiable sampling from categorical distributions.