ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes

The paper addresses the shortage of large labeled RGB-D datasets needed for supervised deep learning in scene understanding, noting that existing datasets cover a small range of scene views with limited semantic annotations. It introduces ScanNet, an RGB-D video dataset containing 2.5 million views across 1513 scenes, annotated with 3D camera poses, surface reconstructions, and semantic segmentations. To collect it, the authors designed an easy-to-use and scalable RGB-D capture system with automated surface reconstruction and crowdsourced semantic annotation.

The richly annotated data enables strong performance on several 3D scene understanding tasks. Using ScanNet, the authors achieve state-of-the-art results on 3D object classification, semantic voxel labeling, and CAD model retrieval, demonstrating that a large, well-annotated RGB-D dataset can drive progress across multiple indoor scene understanding problems.