DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

This work addresses semantic image segmentation with deep learning through three contributions shown to have substantial practical merit. It highlights atrous convolution — convolution with upsampled filters — as a tool for dense prediction that explicitly controls the resolution of feature responses and enlarges filters' field of view without increasing parameters or computation; it proposes atrous spatial pyramid pooling (ASPP), which probes a convolutional feature layer with filters at multiple sampling rates and effective fields of view to robustly segment objects at multiple scales; and it combines the final DCNN layer's responses with a fully connected Conditional Random Field to recover object boundary localization lost to max-pooling and downsampling.

The proposed DeepLab system set a new state of the art on the PASCAL VOC-2012 semantic segmentation task, reaching 79.7 percent mIOU on the test set, and advanced results on PASCAL-Context, PASCAL-Person-Part, and Cityscapes, with the CRF shown both qualitatively and quantitatively to improve localization performance. All of the code was made publicly available online.