DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

DeCAF investigates whether the features produced by the activations of a deep convolutional network, trained in a fully supervised fashion on a large fixed set of object recognition tasks, can be repurposed for novel generic tasks that may differ significantly from the originally trained tasks and where labeled or unlabeled data is insufficient to conventionally train or adapt a deep architecture. The authors investigate and visualize the semantic clustering of these deep convolutional features across a variety of tasks and compare the efficacy of relying on various network levels to define a fixed feature.

Relying on features from different network layers, the transferred representations produce novel results that significantly outperform the state of the art on several important vision challenges, including scene recognition, domain adaptation, and fine-grained recognition. The authors release DeCAF as an open-source implementation with all associated network parameters, enabling vision researchers to experiment with deep representations across visual concept learning paradigms and helping catalyze the widespread use of pretrained deep features.