The Cityscapes Dataset for Semantic Urban Scene Understanding

Noting that no existing dataset adequately captures the complexity of real-world urban scenes for semantic urban scene understanding, the authors introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. The dataset comprises a large, diverse set of stereo video sequences recorded in streets from 50 different cities, with 5000 images carrying high-quality pixel-level annotations and 20000 additional images with coarse annotations to enable methods that leverage large volumes of weakly-labeled data.

The effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. An accompanying empirical study provides an in-depth analysis of the dataset's characteristics as well as a performance evaluation of several state-of-the-art approaches on the benchmark, filling the gap left by object detection datasets that had fueled deep learning progress but did not capture urban scene complexity.