Network In Network

The paper proposes Network In Network (NIN), a novel deep network structure that enhances model discriminability for local patches within the receptive field. Whereas a conventional convolutional layer applies linear filters followed by a nonlinear activation to scan the input, NIN instead builds micro neural networks—instantiated as multilayer perceptrons, which are potent function approximators—to abstract the data within the receptive field. These micro networks are slid over the input like a CNN to produce feature maps, and deep NIN is formed by stacking multiple such structures.

Because the micro networks provide enhanced local modeling, NIN can use global average pooling over feature maps in the classification layer, which the authors argue is easier to interpret and less prone to overfitting than traditional fully connected layers. NIN demonstrated state-of-the-art classification performance on CIFAR-10 and CIFAR-100, along with reasonable performance on the SVHN and MNIST datasets, influencing later architectures that adopted its ideas.