Deformable DETR: Deformable Transformers for End-to-End Object Detection

DETR was proposed to eliminate many hand-designed components in object detection, but it suffers from slow convergence and limited feature spatial resolution, because Transformer attention modules are inefficient at processing image feature maps. Deformable DETR mitigates these problems by using attention modules that attend only to a small set of key sampling points around a reference point, combining the end-to-end nature of DETR with more efficient, localized attention.

Deformable DETR achieves better detection performance than DETR, especially on small objects, while needing 10x fewer training epochs. Extensive experiments on the COCO benchmark demonstrate the effectiveness of the approach, showing that deformable attention resolves DETR's main convergence and resolution bottlenecks.