EfficientNetV2: Smaller Models and Faster Training

EfficientNetV2 introduces a new family of convolutional networks designed for faster training speed and better parameter efficiency than previous models. To build them, the authors use a combination of training-aware neural architecture search and scaling to jointly optimize training speed and parameter efficiency, searching from a space enriched with new operations such as Fused-MBConv. They also train with progressively increasing image size, and because this can reduce accuracy, they propose adaptively adjusting regularization such as dropout and data augmentation as the image size grows.

Experiments show EfficientNetV2 models train much faster than state-of-the-art models while being up to 6.8x smaller. With progressive learning, they significantly outperform prior models on ImageNet and on CIFAR, Cars, and Flowers datasets, and after pretraining on ImageNet21k, EfficientNetV2 attains 87.3% top-1 accuracy on ImageNet ILSVRC2012, exceeding a recent Vision Transformer by 2.0% while training 5x to 11x faster using the same computing resources.