Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

This paper addresses the bottleneck in object detection pipelines caused by separate, slow region proposal algorithms. The authors propose a Region Proposal Network (RPN), a fully convolutional network that shares full-image convolutional features with a downstream Fast R-CNN detector, simultaneously predicting object bounding boxes and objectness scores at each spatial position with almost no added computation, and merge the RPN and detector into a single end-to-end trainable network.

The combined Faster R-CNN system achieves a frame rate of 5fps on a GPU using the very deep VGG-16 model while attaining state-of-the-art object detection accuracy on PASCAL VOC 2007, VOC 2012, and MS COCO using just 300 proposals per image. This efficiency and accuracy made Faster R-CNN and its RPN component the basis for the first-place winning entries across several tracks in the ILSVRC and COCO 2015 competitions, cementing it as a foundational object detection architecture.