Microsoft COCO: Common Objects in Context

This paper introduces the Microsoft COCO dataset, built to advance object recognition by framing it within the broader problem of scene understanding rather than isolated object labeling. The dataset consists of images of complex, everyday scenes containing 91 common object categories recognizable by a young child, with objects labeled using per-instance segmentation masks to support precise localization, and it was assembled through extensive crowd-worker annotation via novel interfaces for category detection, instance spotting, and segmentation.

The resulting dataset contains 2.5 million labeled object instances across 328,000 images, making it substantially larger and richer in contextual annotation than prior benchmarks. The authors provide a detailed statistical comparison against PASCAL, ImageNet, and SUN, and report baseline bounding-box and segmentation detection results using a Deformable Parts Model, establishing COCO as a benchmark that pushed the field toward context-rich, instance-segmented object recognition.