A Survey on Vision Transformer

This survey reviews the growing body of vision transformer models, which adapt the self-attention-based transformer, originally from natural language processing, to computer vision tasks. It organizes these models into categories spanning backbone networks, high and mid-level vision, low-level vision, and video processing, analyzing the advantages and disadvantages of each. The paper also examines the self-attention mechanism as the core component underlying transformers and reviews efficient transformer methods aimed at pushing these models onto real devices.

The survey highlights that transformer-based models perform comparably to or better than convolutional and recurrent networks across many visual benchmarks, while requiring less vision-specific inductive bias. This combination of high performance and reduced reliance on hand-designed priors explains why transformers drew rapid attention from the computer vision community, and the paper closes by discussing outstanding challenges and promising future research directions for the field.