Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

The paper presents Swin Transformer, a new vision Transformer designed to serve as a general-purpose backbone for computer vision. It addresses core challenges in porting Transformers from language to vision, namely the large scale variation of visual entities and the much higher resolution of image pixels compared to words in text. The core method is a hierarchical Transformer architecture built on shifted windows: self-attention is computed within non-overlapping local windows for efficiency, while a shifting scheme across layers still allows connections between windows, giving the model linear computational complexity with respect to image size and the flexibility to model features at multiple scales.

Swin Transformer achieves 87.3% top-1 accuracy on ImageNet-1K image classification, 58.7 box AP and 51.1 mask AP on COCO object detection and instance segmentation, and 53.5 mIoU on ADE20K semantic segmentation, surpassing the prior state of the art by large margins (+2.7 box AP, +2.6 mask AP, +3.2 mIoU). This mattered because it showed a single hierarchical, efficient Transformer architecture could serve as a strong backbone across classification and dense prediction tasks alike, and the hierarchical shifted-window design also benefited all-MLP architectures, broadening its applicability.