Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models.