Worldwide AI Ethics: a review of 200 guidelines and recommendations for AI governance

🔬 Research Summary by Nicholas Kluge Corrêa, a Ph.D. student and a Master’s in Electrical Engineering from the Pontifical Catholic University of Rio Grande do Sul (PUC-RS)/Bonn University. He is the president of the PUC-RS chapter of the AI Robotics Ethics Society. He works as a Machine Learning Engineer in areas related to AI Ethics, AI Safety, and AI Alignment.

[Original paper by Nicholas Kluge Corrêa, Camila Galvão, James William Santos, Carolina Del Pino, Edson Pontes Pinto, Camila Barbosa, Diogo Massmann, Rodrigo Mambrini, Luiza Galvão, Edmund Terem, Nythamar de Oliveira]

Overview: To determine whether a global consensus exists regarding the ethical principles that should govern AI applications and to contribute to the formation of future regulations, this paper conducts a meta-analysis of 200 governance policies and ethical guidelines for AI usage published by different stakeholders worldwide.

Introduction

Given the rising hype surrounding AI technologies, much attention has been given to the question of “What are the values that should guide the development of these systems?”. From moratoriums to the AI Industry to endless lists of ethical principles, it is easy to get lost in this normative discourse if you are new to the field.

Thus, motivated by past research and seeking to bring more clarity to the field, we sought to collect a large sample of AI guidelines and analyze their content, with the aim of finding, in a descriptive manner, “Is there some consensus regarding the previous questions?”. To let the reader make his conclusions, we made our dataset available, paired with an easy-to-use and open-source visualization tool that (in our opinion) surpasses anything produced thus far by past meta-analysts of the field.

Our analysis shows many interesting trends, such as the 2018 AI ethics boom, the persistent lack of mention of sustainability and labor rights-related principles, the soft and unbinding way some regulations are proposed, and much more.

Key Insights

Welcome to Worldwide AI Ethics

After the AI winter in the late 80s and early 90s, AI research and industry experienced remarkable growth. For example, the rise of computer science-related articles on platforms like arXiv indicates a tenfold increase in submissions since 2018, and investment in AI companies has skyrocketed, exceeding $90 billion in the US alone in 2021. With this growth, the “ethical side” grew too, and much work is being done to define the values that should guide these advances. However, one challenge lies in establishing a consensus on these values, given the diverse perspectives of various stakeholders worldwide and the plain abstraction of normative discourse.

What have we done?

To shed light on this, we have meticulously analyzed over 200 ethical guidelines from different sources and languages. Our ultimate goal was to determine the most advocated principles and assess if there’s a consistent understanding across the global AI community while mapping trends that might help researchers paint a representative picture of the field.

Our research also addresses gaps in previous research that preceded and inspired our own. For example, we’ve expanded the sample size significantly, including 200 documents from 37 countries across six continents, covering five languages. Secondly, we’ve introduced a more detailed document typology to go beyond quantity and delve into content analysis. Thirdly, we’ve presented the findings in a user-friendly data visualization framework, making it easier to grasp the information. Lastly, we’ve released an open-source dataset, allowing others to reproduce and build upon our work.

Limitations

While we’ve done our best to shed light on AI ethics discourse, we want to be upfront about the limitations we faced during our analysis. One of these is the small sample size (but larger than any previous one), which means our findings only scratch the surface of the global AI landscape. Also, our language bias limited our work since we couldn’t cover all relevant perspectives due to our linguistic limitations. At the same time, focusing on published guidelines made us miss valuable insights from ongoing discussions in other forms of media, like academic papers, which we did not include in our sample. We also recognize that our “male/female” gender analysis falls short in addressing gender inequality and other related concerns that are extremely present in all aspects of society.

Some Findings

Let’s dive into some insights from our analysis.

Regarding the distribution of produced documents by world regions/countries, countries like the UK and Germany take the lead in publications in Europe. At the same time, the North American continent (with the USA and Canada) holds the lead worldwide regarding the number of published guidelines, with Asia, South America, Africa, and Oceania trailing behind.

Switching our gaze to institution types, besides big organizations like IBM, Microsoft, and UNESCO, most other institutions didn’t go beyond publishing two documents. Also, governmental institutions and private corporations take the lead, accounting for almost half of the documents of our sample (both with equal participation). CSOs/NGOs and non-profit organizations follow behind, with academic institutions trailing at 12.5%. However, this pattern changes depending on where you look on the globe. For example, in North America (mainly the USA), private corporations and non-profit organizations lead the cluster, with governmental institutions following suit, a trend not seen in Europe.

As a brief last mention, let us focus on the gender disparity. 66% of our samples didn’t provide any authorship info. But among the remaining documents, we noticed that most authors with identifiable names were male, making up 66% of the dataset, while female authors represented 34%. Even in academic institutions and non-profit organizations, where the gap is relatively smaller, we’re still nothing close to the 1:1 gender ratio we should have.

Between the lines

Many other results are mentioned in our paper, while many others remain hiding in our visualization tool, waiting to be discovered.

In the end, the main delivery of this work is a tool that can help researchers and policymakers explore the content of AI Ethics Guidelines interactively. For example, while other works give a static picture of the field, ours can answer more complex and interrelated ones:

What is the principle most defended by private corporations worldwide? (Reliability).
What principle possesses the least practical documents proposing solutions for their problems in North America? (Children & Adolescents’ Rights).
Which country is at the front proposing legally binding regulations aided by practical tools to assist AI developers? (United Kingdom).

Thus, we hope this research (limited as it is) can be used as a stepping stone for future works. So, Don’t forget to check our visualization tool!