Fully-Convolutional Siamese Networks for Object Tracking

Object tracking has traditionally relied on learning a model of the target's appearance exclusively online, using only the video itself as training data, which limits how rich the learned model can be. Attempts to harness deep convolutional networks run into a bottleneck: when the target is unknown in advance, online stochastic gradient descent is needed to adapt the weights, severely compromising speed. This paper instead equips a basic tracking algorithm with a fully-convolutional Siamese network trained end-to-end offline on the ILSVRC15 video object detection dataset.

Because the matching function is learned ahead of time, the tracker avoids expensive online adaptation and operates at frame-rates beyond real-time. Despite its extreme simplicity, it achieves state-of-the-art performance across multiple tracking benchmarks, demonstrating that a pre-trained similarity function can drive fast and accurate tracking.