Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding

Neural networks are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources. Deep compression addresses this with a three-stage pipeline: the network is first pruned by learning only the important connections, weights are then quantized to enforce weight sharing, and finally Huffman coding is applied; after the first two stages the network is retrained to fine-tune the remaining connections and the quantized centroids.

The method reduces storage requirements by 35x to 49x without affecting accuracy: on ImageNet, AlexNet shrinks from 240MB to 6.9MB and VGG-16 from 552MB to 11.3MB. Pruning reduces connections 9x to 13x and quantization cuts bits per connection from 32 to 5, allowing models to fit into on-chip SRAM cache rather than off-chip DRAM; benchmarked on CPU, GPU, and mobile GPU, compressed networks show 3x to 4x layerwise speedup and 3x to 7x better energy efficiency, facilitating complex neural networks in mobile applications constrained by size and bandwidth.