Deformable Convolutional Networks

Standard CNNs struggle to model geometric transformations because their building modules use fixed geometric structures and sampling grids. The paper introduces two new modules, deformable convolution and deformable RoI pooling, both based on augmenting the spatial sampling locations with additional offsets. These offsets are learned directly from the target task without any additional supervision, and the modules can replace their plain counterparts in existing CNNs and be trained end-to-end with standard back-propagation, giving rise to deformable convolutional networks.

Extensive experiments validate the approach and, for the first time, demonstrate that learning dense spatial transformation inside deep CNNs is effective for sophisticated vision tasks such as object detection and semantic segmentation. This mattered because it showed geometric adaptivity could be learned rather than hand-designed, and the authors released their code publicly.