Explanation

Full Title or Meme

Artificial Intelligence engines are being asked for an Explanation of how they came to a decision. Does that make any sense?

Problems

• It is well known that users are likely to leave checkboxes as they were if that speeds up the process. For example, both the Dutch and the Belgium driver's license show a check box for organ donation. The Dutch box is initially unchecked, the Belgian box is initially checked. The Dutch acceptance of organ donation is 27%, the Belgium acceptance in 98%. When people were asked to make an Explanation their choice, they typically were able to make a cogent explanation of their "choice". What possible value could that explanation be to understanding?
• For ethical and safety reason, users often expect an explanation of how the networks came to a conclusion in medical, financial, legal and military applications.

Solutions

• The only possible explanation would be one that was made at the same time as the choice, but even then, as Malcom Gladwell explained in his book Blink the choice might have been make very early and the explanation later.
• In Daniel Kahneman's work blink is called "system 1" which, like neural networks, makes rapid, heuristic, decisions, and the more rigorous and methodic "system 2" comes into play, perhaps after the decision is made and "second thoughts" come into play.
• Yann LeCun, chief AI scientist considers widespread concerns about explanations about how an AI works to be a "non-issue". If user Adopt "self-supervised learning, things that are not trained for a give task, but are trained generically, the problems disappear. This sounds like the arrogance of the technologist.
• Symbolic representations of decisions seems like a possibility, although LeCun went on to say "Absolutely not! I don't believe in this at all. Humans don't have discrete symbols. We have patterns of activities in neurons." Funny, it seems that all of human communication is in terms of discrete symbols that we call words or even gestures.