Representation Learning with Contrastive Predictive Coding

The paper proposes Contrastive Predictive Coding, a universal unsupervised learning approach for extracting useful representations from high-dimensional data. The key insight is to learn representations by predicting the future in latent space using powerful autoregressive models, with a probabilistic contrastive loss that induces the latent space to capture information maximally useful for predicting future samples; negative sampling keeps the model tractable.

While most prior work evaluated representations for a particular modality, this approach learns useful representations that achieve strong performance on four distinct domains: speech, images, text, and reinforcement learning in 3D environments, showing that unsupervised representation learning can be broadly applicable rather than modality-specific.