Recent advances in convolutional neural networks

This paper is a broad survey of recent advances in convolutional neural networks, situated in the context of deep learning's strong performance on problems such as visual recognition, speech recognition, and natural language processing. The authors note that among deep neural network types, CNNs have been the most extensively studied, and that their rapid progress has been enabled by the growth in annotated data and major improvements in graphics processing units. The survey details CNN improvements across several aspects, including layer design, activation functions, loss functions, regularization, optimization, and fast computation.

Beyond architectural and training components, the survey introduces various applications of convolutional neural networks in computer vision, speech, and natural language processing. By organizing the many recent developments into a coherent overview, the paper served as a reference that consolidated the state of CNN research and its applications for practitioners and researchers.