Why Machine Learning Explanations Disagree and How to Interpret Them
Summary
This article examines why explainability methods can give conflicting accounts of the same machine learning model. It distinguishes global explanations, which describe overall feature importance or behavior, from local explanations focused on an individual prediction. It also contrasts model-agnostic explainers with methods tailored to a particular model, and describes perturbation-based and gradient-based approaches. A worked example predicts athlete salaries from player attributes using several model families, then compares explanations; the discussion also connects these issues to applying machine learning to market data.
The article emphasizes that explainers rely on different assumptions and can behave differently with data structure, model form, and feature interactions. It cautions that explanations may all be misleading, so resolving a disagreement is not always possible or worthwhile. The example is illustrative rather than evidence of trading utility, and the text does not establish a universal procedure for ranking explanations. It recommends considering interpretable models such as generalized additive models or explainable boosting machines where practical.
Key ideas
- Global explanations summarize model behavior, while local explanations address an individual prediction.
- Model-agnostic and model-specific explainers can differ in assumptions and fidelity.
- Perturbation and gradient methods assess feature influence through different mechanisms.
- Data structure, model choice, and feature interactions can contribute to conflicting explanations.
- Explanations can be unreliable, so interpretable models may be preferable in some applications.
Tags
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.