Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

SwAV is an online self-supervised representation learning algorithm that captures the benefits of contrastive methods without their expensive explicit pairwise feature comparisons. Instead of comparing features directly, it simultaneously clusters the data while enforcing consistency between the cluster assignments produced for different augmentations, or views, of the same image. This is done through a swapped prediction mechanism, in which the cluster assignment of one view is predicted from the representation of another. The authors also introduce a multi-crop augmentation strategy that mixes views of different resolutions without much added memory or compute.

Because it avoids pairwise comparisons, SwAV works with both large and small batches, scales to unlimited data, and is more memory efficient than prior contrastive methods, requiring no large memory bank or momentum network. It achieves 75.3% top-1 accuracy on ImageNet with a ResNet-50 and surpasses supervised pretraining on all considered transfer tasks, marking strong progress in closing the gap between unsupervised and supervised visual representation learning.