A Unified Approach to Interpreting Model Predictions

Understanding why a model makes a particular prediction can be as important as the prediction's accuracy in many applications, yet the highest accuracy on large modern datasets is often achieved by complex models such as ensembles or deep learning models that even experts struggle to interpret. Various methods have recently been proposed to help interpret complex model predictions, but it is often unclear how they relate to one another or when one is preferable. The paper addresses this by presenting SHAP (SHapley Additive exPlanations), a unified framework that assigns each feature an importance value for a particular prediction.

SHAP's novel components include identifying a new class of additive feature importance measures and proving there is a unique solution within this class possessing a set of desirable properties. This unifies six existing methods, notably showing that several recent methods in the class lack the proposed desirable properties, and based on insights from this unification the authors present new methods that show improved computational performance and better consistency with human intuition than previous approaches.