CNN Features Off-the-Shelf: An Astounding Baseline for Recognition

This paper adds to the mounting evidence that generic descriptors extracted from convolutional neural networks form an excellent image representation. Using the publicly available code and model of the OverFeat network, trained for object classification on ILSVRC13, the authors run a series of experiments across recognition tasks that gradually move further away from the original training task and data, including object image classification, scene recognition, fine-grained recognition, attribute detection, and image retrieval.

Using a simple linear SVM classifier (or L2 distance for retrieval) on a 4096-dimensional feature extracted from a layer of the network, further modified with simple augmentation such as jittering, the method consistently achieves superior results to highly tuned state-of-the-art systems across the visual classification tasks, and for instance retrieval outperforms low memory footprint methods except on a sculptures dataset. The results strongly suggested deep convolutional features should be the primary candidate for most visual recognition tasks.