MLP-Mixer: An all-MLP Architecture for Vision

MLP-Mixer proposes a computer vision architecture built exclusively from multi-layer perceptrons, dispensing with both the convolutions of CNNs and the self-attention of Vision Transformers. The architecture uses two distinct types of layers: one applies MLPs independently to image patches to mix the per-location (channel) features, and the other applies MLPs across patches to mix spatial information between locations. This design demonstrates the paper's central claim that while convolutions and attention are each sufficient for good performance, neither of them is actually necessary.

When trained on large datasets or combined with modern regularization schemes, MLP-Mixer attains competitive scores on image classification benchmarks, with pre-training and inference costs comparable to state-of-the-art convolutional and transformer models. The result mattered because it challenged the assumed necessity of convolutions and attention for strong visual performance, and the authors hoped it would spark further research beyond the well-established realms of CNNs and Transformers.