Learning Transferable Architectures for Scalable Image Recognition

Developing neural networks for image classification usually requires substantial manual architecture engineering, so the authors study how to learn model architectures directly on the dataset of interest. Because such a search is prohibitively expensive on large datasets, they instead search for a small architectural building block, or 'cell,' on the modest CIFAR-10 dataset and then transfer it to larger datasets by stacking multiple copies, each with its own parameters. The key enabler is a newly designed, transferable search space they call the 'NASNet search space,' complemented by a new regularization technique, ScheduledDropPath, that markedly improves generalization.

The best cell found on CIFAR-10 achieves a state-of-the-art 2.4% error rate there, and a NASNet built by stacking that cell reaches 82.7% top-1 and 96.2% top-5 accuracy on ImageNet, 1.2% better in top-1 than the best human-designed architectures while using 9 billion fewer FLOPS, a 28% reduction in computational demand. Smaller NASNets also outperform comparably sized mobile models, and the learned features transfer well to object detection, where combining them with Faster R-CNN yields 43.1% mAP on COCO, surpassing the prior state of the art by 4.0%.