Summary contributed by Nga Than (@NgaThanNYC), a doctoral candidate in the Sociology program at City University of New York – The Graduate Center.
[Link to original paper + authors at the bottom]
Overview: This paper outlines the role of “ethics owners,” a new occupational group in the tech industry, whose jobs are to examine ethical consequences of technological innovations. The authors highlight competing logics that they have to navigate, and two ethical pitfalls that might result from those different imperatives.
Google’s recent firing of Timnit Gebru, a prominent AI ethics researcher, has shaken the AI ethics community. The event has called into question two main issues in the tech industry: its lack of diversity, and its faulty relationship with ethical considerations of technological development. Dr. Gebru represents the growing “ethics owners class of tech workers” who champion ethical causes, ethical designs, development, and deployment of technology from within the tech industry. This research article provides a conceptual framework to understand this emerging occupation and the various day-to-day struggles that these ethics professionals like Dr. Gebru are facing within the industry.
The authors employed a mixed qualitative methods approach by gathering ethnographic, textual and interviewing data. They interviewed 17 ethics professionals from different well-known companies whose formal roles have been to address ethics within their companies or within the industry. This particular role has become institutionalized after a series of public scandals such as Cambridge Analytica in the 2016 US presidential election, racial biases in facial recognition technology broke out.
Ethics owners’ daily activities are to examine the social consequences of technology products. Their jobs are similar, but not the same as the business ethics, legal ethics, and sometimes PR teams. They respond to external pressures to tech companies within corporate boundaries. Sometimes they are seasoned engineers, MBA holders, trained in social sciences, or humanities. This class of tech workers are trying to change the tech industry from within by “fulfilling fundamental ethics commitments.” The article defines the process of institutionalization of ethics “as a set of roles, and responsibilities,” and “operationalized as a set of practices and procedures.” They argue that ethics owners “operate inside a fraught dynamic.” While “attempting to resolve critical external normative claims about the core logics of the tech industry,” they have to do so within the corporate structure, and being embedded within corporate logics. This might lead to structural, cultural and social pitfalls.
The authors use the ethnographic approach to ethics or the “ordinary ethics approach.” Instead of thinking about ethics as a set of abstract concepts, and principles, they examine “how ethics and morality structure social life,” and “how everyday practices reveal the moral commitments embedded in actions.” They found that “ethics as everyday practice” meets with challenges because tech workers, managers, and other stakeholders are not necessarily aware of ethics. Inside these companies, ethics owners are in charge of developing “strategies to align everyday practices with corporate logics” while navigating the everydayness of corporate life. They actively define, and help their companies to locate where ethics responsibility lies within the organizational hierarchy.
The role of ethics owners within the tech industry is ambivalent. Ethics owners operating within the industry are up against corporate logics that might prevent them from implementing their work or from achieving intended results. The three main corporate and industry logics that the authors examine are meritocracy, technological solutionism, and market fundamentalism.
Meritocracy is an ideological framework that legitimizes unequal distributions of wealth and power as arising from differences in individual abilities. This has defined the modern subject: as autonomous and responsible for perpetual self-improvement. The tech industry was founded on the myth that it is a meritocratic segment where talents should be rewarded handsomely. This meritocratic belief manifests in the idea that engineers are best at solving ethical issues that their products might create. Similarly, meritocratic logics place a strong emphasis on individual ethics rather than regulation and legislation. Companies and teams try to come up with their own codes of ethics to drive off legislation. The authors conclude that despite their best efforts, ethics owners’ perspectives on larger societal problems are partial, as are their roles within the industry.
Technological solutionism is the belief that technology can solve social problems, which are then reinforced by the financial rewards that the industry has gained for producing technology that they believe solve the problems. Critics have pointed out that many so-called “solutions” can actually cause problems such as rising income and housing inequalities. The tech industry often responds by proposing even more technical solutions. Similarly, ethical problems are also framed as could be solved by technological solutions. This logic leads to creation of checklists, procedures or evaluative metrics to ensure the design and implementation of ethical products. The authors however point out that this approach is limited, and problematic because it centers ethics in the practices of technologists, and not in the social worlds wherein technical systems are created.
Market Fundamentalism, or market logics, refers to the idea that companies are there to make money, and if ethics initiatives are cut into the bottom line, companies should not do it. Besides, there is a belief that ethical initiatives are often costly, and antithetical to corporate profits. Furthermore across the industry, if other companies do not implement similar ethical considerations on their products, one should not do it. In the context of the absence of a legal framework, implementing ethics initiatives might be a business problem rather than a solution. In other words, the works of ethics owners in practice are constrained by what the market can allow.
These three different corporate logics reinforce each other and create a dynamic in which ethics owners have to navigate. Operating within these different logics can create scenarios which the authors termed “normalizing ethical mishaps,” and “blinkered isomorphism.” Normalizing ethical mishaps refers to situations when tech companies create structures that “normalize ethical transgressions;” while “blinkered isomorphism” refer to the process whereby tech companies converge to one structure by learning from each other’s extreme cases while overlooking everyday ethical failings.
The article sheds light to the current events around the departure of Dr. Gebru from Google. The research shows ethics owners such as Dr. Gebru have to negotiate different competing logics between corporate interests, personal, and professional commitments. The recent events seem to suggest that when corporate logics appear to be more important, ethics owners regardless of how prominent they are could be let go. In other words, “ethics owners” occupy both ambivalent and precarious positions within the tech industry hierarchy.
Original paper by Jacob Metcalf, Emmanuel Moss, and danah boyd: https://datasociety.net/wp-content/uploads/2019/09/Owning-Ethics-PDF-version-2.pdf