An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

This paper challenges the assumption that convolutional networks are necessary for computer vision. The authors split images into fixed-size patches, treat the sequence of patches like tokens in NLP, and feed them directly into a standard Transformer architecture with no convolutional layers, calling the resulting model the Vision Transformer (ViT).

When pre-trained on large amounts of data and then transferred to mid-sized or small image recognition benchmarks such as ImageNet, CIFAR-100, and VTAB, ViT attains excellent results compared to state-of-the-art convolutional networks, while requiring substantially fewer computational resources to train. This demonstrated that a pure attention-based architecture, given sufficient pretraining data, can rival or surpass CNNs on core vision tasks, reshaping the design space for vision models.