Adversarial examples in the physical world

The paper studies adversarial examples—input samples modified very slightly, often so subtly that humans do not notice, in ways designed to make a machine learning classifier misclassify them. It notes these pose security concerns and that all prior work assumed a threat model in which the adversary can feed data directly into the classifier. That assumption fails for systems operating in the physical world, such as those taking input from cameras and other sensors, which motivates the paper's investigation.

To test physical-world robustness, the authors obtain adversarial images, capture them through a cell-phone camera, and feed them to an ImageNet Inception classifier, measuring accuracy. They find that a large fraction of adversarial examples are still classified incorrectly even when perceived through the camera. This demonstrated that adversarial attacks can threaten machine learning systems even without direct digital access to their inputs, which mattered for the security of real-world sensor-based systems.