Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI

[Original paper by Michael A. Madaio, Jennifer Wortman Vaughan, Luke Stark and Hanna Wallach (2020)]

---

Overview: Among the burgeoning literature on AI ethics and the values that would be important to respect in the development and use of artificial intelligence systems (AIS), fairness comes up a few times, perhaps as an echo of the very current notion of social justice. Authors Madaio, Vaughan, Stark and Wallach (Microsoft Research) have co-developed a checklist that seeks to ensure fairness, while recognizing that a procedure alone cannot overcome the value tensions and incompatibilities in ethical practice.

---

Introduction

Anyone interested in the ethics of artificial intelligence is aware of this: a plethora of position papers on AI ethics have emerged in the last five years. They come from private companies, civil society, universities, as well as governmental and international organizations.

Authors Michael A. Madaio, Jennifer Wortman Vaughan, Luke Stark, and Hanna Wallach (Microsoft Research) noted this sort of buzz around AI ethics, while remarking that the level of abstraction of many of these statements posed problems for their practical application (p.1).

To avoid this pitfall, these researchers participated in the development of an equity checklist with 48 AI practitioners from a dozen companies working in a variety of AI applications (pp.1, 5). Through semi-structured interviews as well as “[…] an iterative co-design process […]” (p.1), the authors were guided by three research questions:

“RQ1: What are practitioners’ current processes for identifying and mitigating AI fairness issues?

RQ2: What are practitioners’ desiderata and concerns regarding AI fairness checklists?

RQ3: How do practitioners envision AI fairness checklists might be implemented within their organizations?” (emphasis in the text, p.4)

Key Insights

The disconnect between principles and practice is a criticism that has been repeatedly leveled at AI ethical guidelines. Madaio et al. obviously intended to avoid this pitfall.

But how do we avoid this gap between ethics and technical practice (p.1)? Even with all the goodwill in the world, checklists for ethical AI development and deployment may be poorly followed by practitioners or ignored altogether. Even more, the items on the checklist may prove incompatible in practice (p.2), conflicting in potentially irreparable ways.

The authors are well aware of this, so much so that they admit that “[…] AI ethics principles can place practitioners in a challenging moral bind by establishing ethical responsibilities to different stakeholders without offering any guidance on how to navigate tradeoffs when these stakeholders’ needs or expectations conflict.” (p.2)

Would the solution to this challenge of compromise lie in a “technologization” of ethics? Madaio et al. do not think so (p.2.) One should not imagine that a simple answer to a binary question (p.3) captures the complexity of the ethical dilemma in which the developer may find himself.

Several contextual elements must be considered, for example the sector of activity or research (public or private), or the size of the company (p.10). There is often disagreement about the definition of the concepts themselves (p.3). In the context of developing their equity checklist, Madaio et al. explicitly acknowledge that the very concept of equity can be understood differently in different contexts: “Fairness is a complex concept and deeply contextual [and] […] There is no single definition of fairness that will apply equally well to different applications of AI” (p.15). For example, equity can be understood in personal or organizational terms (p.5).

The checklist proposed by Madaio et al., based on a previously developed checklist model (p.4) and modified according to the co-design workshops with the participants, consists of six main steps, which roughly correspond to the development of an artificial intelligence system (pp.16-20):

1. “Envision”

2. “Define”

3. “Prototype”

4. “Build”

5. “Launch”

6. “Evolve”

At all stages of the checklist, it is necessary to ensure that the criterion of fairness can be respected or, if compromises are necessary, to document them and to consider dropping the project if this would be preferable (pp.16-20). This proposal guarantees a great honesty in the development of AIS, orienting the approach not only towards the maximization of efficiency, but also towards the common good, encapsulated for these authors in the value of equity.

For Madaio et al, dialogue is central to the use of this list (p.16). The discussion must involve stakeholders as diverse as the people who will use the technology, who will be affected by it, the practitioners who develop it, their teams, and experts to consult at different stages of the process. Hence, the authors suggest that “[…] the most beneficial outcome of implementing an AI ethics checklist may be to prompt discussion and reflection that might otherwise not take place.” (p.3)

Another advantage of the checklist, according to the authors, is that it would make it possible to establish a preventive rather than a reactive ethic, which would have the mission of anticipating ethical glitches, while being adapted to the operating modes of the practitioners. With such a tool, we could possibly see a reduction in anxiety among developers (p. 6-7).

That said, the use of the checklist does not guarantee the eradication of equity problems, but their prevention and mitigation, as much as possible (p.15). In other words, one cannot eradicate tensions or value clashes in practice, but one can seek to minimize the negative effects of the trade-off one has found (cf. Blattberg 2018, 151). This view is not unlike the philosophical school of “value pluralists” such as Isaiah Berlin, Bernard Williams, or Stuart Hampshire.

One caveat, noted by the participants in this study, is that the checklist may be used merely as a formal, “minimal” process (p. 8), rather than as a means of generating deep conversations about the ethical implications of the technology being developed (p.8). It is important to clarify the role of the checklist. While it serves as a tool for discussion in the implementation of ethics, it is not understood in a fully procedural way. Indeed, this procedural understanding could be problematic, as one study participant noted: “[…] ‘I’m a little bit suspicious of the checklist approach. I actually tend to think that when we have highly procedural processes we wind up with really procedural understandings of fairness’” (p.8).

The danger is there and, basically, it is difficult to reduce the richness of a concept such as equity to a definition and a procedure. The authors heard these concerns and adapted their model accordingly: “[…] our checklist items are intended to prompt critical conversations, using words like ‘scrutinize’ and asking teams to ‘define fairness criteria’ rather than including specific fairness criteria or thresholds to meet” (p.8).

Between the lines

Some will be skeptical of a proposal such as the Madaio et al.’s checklist, since it does not appear to “fix the problem” of AI ethics once and for all. On the contrary, this checklist would rather refer to “[…] a way to spur ‘good tension,’ prompting critical conversations and prying open discussion about AI fairness […]” (p.10). These conversations will be made possible by the organizational culture – an element that should not be overlooked (p.10).

To try to solve all ethical problems in advance by means of a procedure is probably idealistic. As the authors rightly suggest, “[t]here are seldom clear-cut answers. It is therefore important to document your processes and considerations (including priorities and tradeoffs), and to seek help when needed” (p.15).

Does this mean that AI ethics is, in a certain way, a moving target that no practitioner, theory, or procedure can immobilize once and for all? If that is the case, then dialogue does seem to be a good solution to practice ethics in the specifities of each context. I would add to this that the virtue of prudence would be a good guide for this type of discussion.

Note:

To facilitate the location of the original information, the page numbers for the checklist are those of the pdf document (the continuation of the article).