LabelMe: A Database and Web-Based Tool for Image Annotation

LabelMe addresses the need for a large collection of images with ground-truth labels usable for object detection and recognition research and for quantitative evaluation. The authors developed a web-based tool that makes image annotation easy and allows instant sharing of those annotations, and used it to collect a large dataset that spans many object categories, frequently containing multiple object instances across a wide variety of images.

The paper quantifies the contents of the resulting dataset and compares it against existing state-of-the-art datasets used for object recognition and detection. It further shows how the dataset can be extended: automatically enhancing object labels with WordNet, discovering object parts, recovering a depth ordering of objects in a scene, and increasing the number of labels using minimal user supervision together with images drawn from the web, making it a flexible resource for supervised learning.