Neural Architecture Search with Reinforcement Learning

The paper tackles the difficulty of designing neural networks by automating the process. It uses a recurrent neural network as a controller to generate the model descriptions of candidate neural networks, and trains this controller with reinforcement learning so that it maximizes the expected accuracy of the architectures it produces on a held-out validation set. This turns architecture design into a learned search problem rather than a manual one.

Starting from scratch on CIFAR-10, the method discovers an architecture that rivals the best human-invented networks, reaching a 3.65% test error rate that is 0.09% better and 1.05x faster than the comparable previous state of the art. On the Penn Treebank language modeling task it composes a novel recurrent cell that outperforms the widely used LSTM and other baselines, achieving 62.4 perplexity, and this cell transfers to character-level modeling with a state-of-the-art 1.214 perplexity. The results demonstrated that automated search could match or exceed expert architecture design.