VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection

VoxelNet addresses 3D object detection in sparse lidar point clouds without relying on hand-crafted feature representations such as bird's-eye-view projections. It unifies feature extraction and bounding box prediction into a single-stage, end-to-end trainable deep network: the point cloud is divided into equally spaced 3D voxels, and a newly introduced voxel feature encoding (VFE) layer transforms the group of points within each voxel into a unified feature, yielding a descriptive volumetric representation that connects to a region proposal network to generate detections.

On the KITTI car detection benchmark, VoxelNet outperforms state-of-the-art lidar-based 3D detection methods by a large margin. The network also learns an effective discriminative representation of objects with varied geometries, producing encouraging results for detecting pedestrians and cyclists using only lidar, and demonstrating that manual feature engineering can be replaced by learned voxel encodings for point-cloud detection.