Table 5 Recommendations for designing AI... | Emerald Publishing

Table 5

Recommendations for designing AI system explanations for end users

Recommendation categories	Design recommendation	Reasoning	Sources
General	1. Context is everything – There is no one-size-fits-all type of solution	What to explain is dependent on several factors including what kind of AI system or decision we are explaining, who are the target audience and do we want to optimize for trust, for understandability or do we wish to simply comply by legislation	Bussone et al. (2015), Dodge et al. (2019), Ehsan et al. (2019), Oh et al. (2018), Putnam and Conati (2019), Wang et al. (2019), Xie et al. (2019)
When to explain	2. Provide explanations on demand, not all the time	For certain decisions and in certain moments users' may be interested in seeing more information on AI system decisions. However, constant display of full XAI documentation can hurt the user experience	Chazette and Schneider (2020), Cramer et al. (2008), Lim et al. (2009), Lim and Dey (2009), Oh et al. (2018)
How to explain	3. Personalize explanations	There are various kinds of people with different levels of understanding of AI systems and XAI needs. This could be taken into account when explaining the system	Chazette and Schneider (2020), Cramer et al. (2008), Dodge et al. (2019), Weitz et al. (2019a), Wang et al. (2019), Xie et al. (2019)
	4. Consider visualizing explanations	Users tend to anthropomorphize AI and may benefit from human-like explanations. Visualizing explanations may help some users to accept the AI system and its decisions better	Ngo et al. (2020), Schrills and Franke (2020), Weitz et al. (2019a), Weitz et al. (2019b)
	5. Acknowledge the existence of trade-offs	For example, optimizing explanations for understandability can lead to less details, which can hurt end users' confidence in the explanation	Cheng et al. (2019), Dodge et al. (2019), Ehsan et al. (2019), Weitz et al. (2019a)
	6. Consider potential misconceptions	Users may end up forming or having formed misconceptions regarding the AI system. These may shape behavior and interpretation of explanations in a certain way. Explanations that are able to reshape misconceptions in a constructive way of conceptual change are valuable	Cramer et al. (2008), Oh et al. (2018), Xie et al. (2019)
	7. Link explanations to users' mental models	This makes the AI system easier to understand for end users, increasing transparency	Ngo et al. (2020), Lim et al. (2009)
	8. Strengthen users' curiosity towards the system	To increase user satisfaction especially in creative and learning contexts, provide interesting and even surprising elements to keep the users' curiosity at a high level	Oh et al. (2018), Putnam and Conati (2019)
	9. Ensure the visibility and discoverability of explanations	Make sure AI system end users find and become aware of explanations	Eslami et al. (2018)
	10. Use metaphors to demystify how AI systems work	Metaphors can be more useful in increasing end users' understanding of AI systems than precise but difficult technical language	Ngo et al. (2020)
	11. Support users' own thinking	In professional contexts, such as in medicine, the AI system should provide counterfactuals and explanations so users can reflect on and test their own thinking and hypotheses	Wang et al. (2019)
	12. Provide access to source data	Especially in high-stakes decision making, such as in justice or in medicine, users may want to request access to raw data to build their trust in the AI system	Wang et al. (2019)
	13. Provide users with generalized explanations rather than case-based explanations	Users may consider it quirky if the decision is explained to them with a particular event from the past. To increase user acceptance, refer to generalized past events instead	van der Waa et al. (2020)
What to explain	14. Consider what part of the AI system to explain	Depending on the situation, users may wish to know more about, for example: (1) inputs; (2) outputs; (3) application; (4) situation; (5) model; (6) certainty; and (7) control	Broekens et al. (2010), Lim and Dey (2009)
	15. Explain unfavorable decisions	Users are likely to demand explanations when they disagree with the system	Putnam and Conati (2019)
	16. Communicate the uncertainties involved in the system's decision making	If there is a mismatch between users' expectations of the AI system and its actual capabilities, it hinders users' acceptance and trust building in it. Users should understand the risks of the AI system's making errors	Brennen (2020), Wang et al. (2019), Yin et al. (2019)

Recommendation categories	Design recommendation	Reasoning	Sources
General	1. Context is everything – There is no one-size-fits-all type of solution	What to explain is dependent on several factors including what kind of AI system or decision we are explaining, who are the target audience and do we want to optimize for trust, for understandability or do we wish to simply comply by legislation	Bussone et al. (2015), Dodge et al. (2019), Ehsan et al. (2019), Oh et al. (2018), Putnam and Conati (2019), Wang et al. (2019), Xie et al. (2019)
When to explain	2. Provide explanations on demand, not all the time	For certain decisions and in certain moments users' may be interested in seeing more information on AI system decisions. However, constant display of full XAI documentation can hurt the user experience	Chazette and Schneider (2020), Cramer et al. (2008), Lim et al. (2009), Lim and Dey (2009), Oh et al. (2018)
How to explain	3. Personalize explanations	There are various kinds of people with different levels of understanding of AI systems and XAI needs. This could be taken into account when explaining the system	Chazette and Schneider (2020), Cramer et al. (2008), Dodge et al. (2019), Weitz et al. (2019a), Wang et al. (2019), Xie et al. (2019)
	4. Consider visualizing explanations	Users tend to anthropomorphize AI and may benefit from human-like explanations. Visualizing explanations may help some users to accept the AI system and its decisions better	Ngo et al. (2020), Schrills and Franke (2020), Weitz et al. (2019a), Weitz et al. (2019b)
	5. Acknowledge the existence of trade-offs	For example, optimizing explanations for understandability can lead to less details, which can hurt end users' confidence in the explanation	Cheng et al. (2019), Dodge et al. (2019), Ehsan et al. (2019), Weitz et al. (2019a)
	6. Consider potential misconceptions	Users may end up forming or having formed misconceptions regarding the AI system. These may shape behavior and interpretation of explanations in a certain way. Explanations that are able to reshape misconceptions in a constructive way of conceptual change are valuable	Cramer et al. (2008), Oh et al. (2018), Xie et al. (2019)
	7. Link explanations to users' mental models	This makes the AI system easier to understand for end users, increasing transparency	Ngo et al. (2020), Lim et al. (2009)
	8. Strengthen users' curiosity towards the system	To increase user satisfaction especially in creative and learning contexts, provide interesting and even surprising elements to keep the users' curiosity at a high level	Oh et al. (2018), Putnam and Conati (2019)
	9. Ensure the visibility and discoverability of explanations	Make sure AI system end users find and become aware of explanations	Eslami et al. (2018)
	10. Use metaphors to demystify how AI systems work	Metaphors can be more useful in increasing end users' understanding of AI systems than precise but difficult technical language	Ngo et al. (2020)
	11. Support users' own thinking	In professional contexts, such as in medicine, the AI system should provide counterfactuals and explanations so users can reflect on and test their own thinking and hypotheses	Wang et al. (2019)
	12. Provide access to source data	Especially in high-stakes decision making, such as in justice or in medicine, users may want to request access to raw data to build their trust in the AI system	Wang et al. (2019)
	13. Provide users with generalized explanations rather than case-based explanations	Users may consider it quirky if the decision is explained to them with a particular event from the past. To increase user acceptance, refer to generalized past events instead	van der Waa et al. (2020)
What to explain	14. Consider what part of the AI system to explain	Depending on the situation, users may wish to know more about, for example: (1) inputs; (2) outputs; (3) application; (4) situation; (5) model; (6) certainty; and (7) control	Broekens et al. (2010), Lim and Dey (2009)
	15. Explain unfavorable decisions	Users are likely to demand explanations when they disagree with the system	Putnam and Conati (2019)
	16. Communicate the uncertainties involved in the system's decision making	If there is a mismatch between users' expectations of the AI system and its actual capabilities, it hinders users' acceptance and trust building in it. Users should understand the risks of the AI system's making errors	Brennen (2020), Wang et al. (2019), Yin et al. (2019)