Non-local Neural Networks

Convolutional and recurrent operations are building blocks that process one local neighborhood at a time. This paper presents non-local operations as a generic family of building blocks for capturing long-range dependencies: inspired by the classical non-local means method in computer vision, a non-local operation computes the response at a position as a weighted sum of the features at all positions, and the block can be plugged into many computer vision architectures.

On video classification, non-local models compete with or outperform current competition winners on both the Kinetics and Charades datasets, even without any bells and whistles. On static image recognition, non-local models improve object detection, segmentation, and pose estimation on the COCO suite of tasks, showing the building block's broad applicability across vision problems.