Cascade R-CNN: Delving Into High Quality Object Detection

Object detectors require an intersection-over-union (IoU) threshold to label proposals as positive or negative, but a detector trained with a low threshold such as 0.5 tends to produce noisy detections, while simply raising the threshold degrades performance. The paper attributes this to two problems: overfitting during training as positive samples vanish exponentially at higher thresholds, and a mismatch at inference between the IoUs for which a detector is optimal and those of its input hypotheses. Cascade R-CNN is a multi-stage architecture that tackles both by chaining a sequence of detectors trained with progressively increasing IoU thresholds, each stage becoming more selective against close false positives.

The cascade exploits the observation that a detector's output is a good distribution for training the next, higher-quality detector, and its resampling of progressively improved hypotheses ensures every stage has a positive set of equivalent size, mitigating overfitting; the same cascade procedure is applied at inference so hypotheses match each stage's quality. A simple implementation surpasses all single-model object detectors on the challenging COCO dataset, and experiments show consistent gains across different detector architectures regardless of baseline strength, making it a broadly applicable recipe for high-quality detection.