Summary of the recognized challenges for the use of AI in CHAPE application
| # | Requirements | Challenges |
|---|---|---|
| 1 | Human agency and oversight | Could the AI system affect human autonomy by interfering with the (end) user's decision-making process in an unintended way? |
| Did you take safeguards to prevent overconfidence in or overreliance on the AI system for work processes? | ||
| Who is the “human in control” and what are the opportunities for human intervention? | ||
| 2 | Technical robustness and safety | Did you verify how your system would react in unexpected situations or environments? |
| Did you ensure that your system has a sufficient fallback plan in the case of adversarial attacks or other unexpected situations? | ||
| Did you assess whether there is a probable chance that the AI system may cause damage, or harm to users or third parties? Did you assess the likelihood, potential damage, impacted audience and severity? | ||
| Did you assess what level and definition of accuracy would be required in the context of the AI-system and use case? | ||
| Did you verify what harm would be caused if the AI-system makes inaccurate predictions? | ||
| 3 | Privacy and data governance | Did you consider the ways to develop the AI system or train the model without or with minimal use of potentially sensitive or personal data? |
| Did you take measures to enhance privacy, such as encryption, anonymization and aggregation? | ||
| Did you establish oversight mechanisms for data collection, storage, processing and use? | ||
| Did you ensure that people working with data are qualified and required to access the data, and that they have the necessary competencies to understand the details of data protection policy? | ||
| Did you ensure an oversight mechanism to log when, where, how, by whom and for what purpose data was accessed? | ||
| 4 | Transparency | Did you ensure an oversight mechanism to log when, where, how, by whom and for what purpose data was accessed? |
| Why was this particular system deployed in this specific area? | ||
| Did you establish mechanisms to inform (end-)users on the reasons and criteria behind the AI system's outcomes? | ||
| 5 | Diversity, non-discrimination, and fairness | Did you assess and acknowledge the possible limitations stemming from the composition of the used data sets? |
| Did you assess whether the AI system usable by those with special needs or disabilities or those at risk of exclusion? How was this designed into the system and how is it verified? | ||
| Did you assess whether there could be persons or groups who might be disproportionately affected by negative implications? | ||
| Did you consider a mechanism to include the participation of different stakeholders in the AI system's development and use? | ||
| 6 | Societal and environmental wellbeing | Did you assess whether the AI-system encourages humans to develop attachment and empathy or vice versa? |
| Did you assess whether the logic of the AI-system might simplify and polarize public discourse? | ||
| Did you assess whether the AI-system could be used to manipulate or confuse people? | ||
| Did you ensure measures to reduce the environmental impact of your Al-system's life cycle? | ||
| Did you establish mechanisms to measure the environmental impact of the Al-system's development, deployment, and use (for example the type of energy used by the data centres)? | ||
| 7 | Accountability | Did you establish mechanisms that facilitate the system's auditability, such as ensuring traceability and logging of the AI-systems processes and outcomes? |
| Did you carry out a risk or impact assessment of the AI-system, which considers different stakeholders that are (in)directly affected? | ||
| How do you decide on trade-offs between ethical principles? Did you ensure that the trade-off decision was documented? | ||
| Did you establish an adequate set of mechanisms that allows for redress in the case of the occurrence of any harm or adverse impact? |
| # | Requirements | Challenges |
|---|---|---|
| 1 | Human agency and oversight | Could the AI system affect human autonomy by interfering with the (end) user's decision-making process in an unintended way? |
| Did you take safeguards to prevent overconfidence in or overreliance on the AI system for work processes? | ||
| Who is the “human in control” and what are the opportunities for human intervention? | ||
| 2 | Technical robustness and safety | Did you verify how your system would react in unexpected situations or environments? |
| Did you ensure that your system has a sufficient fallback plan in the case of adversarial attacks or other unexpected situations? | ||
| Did you assess whether there is a probable chance that the AI system may cause damage, or harm to users or third parties? Did you assess the likelihood, potential damage, impacted audience and severity? | ||
| Did you assess what level and definition of accuracy would be required in the context of the AI-system and use case? | ||
| Did you verify what harm would be caused if the AI-system makes inaccurate predictions? | ||
| 3 | Privacy and data governance | Did you consider the ways to develop the AI system or train the model without or with minimal use of potentially sensitive or personal data? |
| Did you take measures to enhance privacy, such as encryption, anonymization and aggregation? | ||
| Did you establish oversight mechanisms for data collection, storage, processing and use? | ||
| Did you ensure that people working with data are qualified and required to access the data, and that they have the necessary competencies to understand the details of data protection policy? | ||
| Did you ensure an oversight mechanism to log when, where, how, by whom and for what purpose data was accessed? | ||
| 4 | Transparency | Did you ensure an oversight mechanism to log when, where, how, by whom and for what purpose data was accessed? |
| Why was this particular system deployed in this specific area? | ||
| Did you establish mechanisms to inform (end-)users on the reasons and criteria behind the AI system's outcomes? | ||
| 5 | Diversity, non-discrimination, and fairness | Did you assess and acknowledge the possible limitations stemming from the composition of the used data sets? |
| Did you assess whether the AI system usable by those with special needs or disabilities or those at risk of exclusion? How was this designed into the system and how is it verified? | ||
| Did you assess whether there could be persons or groups who might be disproportionately affected by negative implications? | ||
| Did you consider a mechanism to include the participation of different stakeholders in the AI system's development and use? | ||
| 6 | Societal and environmental wellbeing | Did you assess whether the AI-system encourages humans to develop attachment and empathy or vice versa? |
| Did you assess whether the logic of the AI-system might simplify and polarize public discourse? | ||
| Did you assess whether the AI-system could be used to manipulate or confuse people? | ||
| Did you ensure measures to reduce the environmental impact of your Al-system's life cycle? | ||
| Did you establish mechanisms to measure the environmental impact of the Al-system's development, deployment, and use (for example the type of energy used by the data centres)? | ||
| 7 | Accountability | Did you establish mechanisms that facilitate the system's auditability, such as ensuring traceability and logging of the AI-systems processes and outcomes? |
| Did you carry out a risk or impact assessment of the AI-system, which considers different stakeholders that are (in)directly affected? | ||
| How do you decide on trade-offs between ethical principles? Did you ensure that the trade-off decision was documented? | ||
| Did you establish an adequate set of mechanisms that allows for redress in the case of the occurrence of any harm or adverse impact? |
Source(s): Table created by author
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.