DARTS: Differentiable Architecture Search

DARTS addresses the scalability challenge of neural architecture search by formulating the search itself in a differentiable manner. Rather than applying evolutionary algorithms or reinforcement learning over a discrete and non-differentiable search space, the method relaxes the architecture representation into a continuous form, which allows the architecture to be optimized efficiently using gradient descent.

Across extensive experiments on CIFAR-10, ImageNet, Penn Treebank, and WikiText-2, DARTS discovers high-performance convolutional architectures for image classification and recurrent architectures for language modeling. It does so while being orders of magnitude faster than state-of-the-art non-differentiable techniques, and the authors release their implementation to support further research on efficient architecture search.