The Dangers of Post-hoc Interpretability: Unjustified Counterfactual
Explanations
Abstract
Post-hoc interpretability approaches have been
proven to be powerful tools to generate explanations for the predictions made by a trained blackbox model. However, they create the risk of having explanations that are a result of some artifacts
learned by the model instead of actual knowledge
from the data. This paper focuses on the case of
counterfactual explanations and asks whether the
generated instances can be justified, i.e. continuously connected to some ground-truth data. We
evaluate the risk of generating unjustified counterfactual examples by investigating the local neighborhoods of instances whose predictions are to be
explained and show that this risk is quite high for
several datasets. Furthermore, we show that most
state of the art approaches do not differentiate justi-
fied from unjustified counterfactual examples, leading to less useful explanations