What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?

The paper focuses on the two major kinds of uncertainty a model can represent: aleatoric uncertainty, which captures noise inherent in the observations themselves, and epistemic uncertainty, which reflects uncertainty in the model and can in principle be explained away given enough data. Noting that epistemic uncertainty has traditionally been difficult to model in computer vision, the authors leverage recent Bayesian deep learning tools to present a framework that combines input-dependent aleatoric uncertainty together with epistemic uncertainty, and they study it on per-pixel semantic segmentation and depth regression tasks.

A key consequence of explicitly formulating uncertainty is a set of new loss functions for these tasks that can be interpreted as learned attenuation, which makes the loss more robust to noisy data by down-weighting uncertain observations. Modeling both uncertainty types in this unified way not only clarifies their respective benefits but also yields new state-of-the-art results on segmentation and depth regression benchmarks, showing that principled uncertainty modeling can improve predictive performance as well as reliability.