• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • šŸ‡«šŸ‡·
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

A Snapshot of the Frontiers of Fairness in Machine Learning (Research Summary)

October 5, 2020

Summary contributed by our Artist-in-Residence Falaah Arif Khan. She’s also a Research Fellow in the CVIT Lab at the International Institute of Information Technology.

Link to original paper + authors at the bottom.


Mini-summary: In this succinct review of the scholarship on Fair Machine Learning(ML), Chouldechova and Roth outline the major strides taken towards understanding algorithmic bias, discuss the merits and shortcomings of proposed approaches, and present salient open questions on the frontiers of Fair ML. These include- statistical vs individual notions of Fairness, the dynamics of fairness in socio-technical systems, and the detection and correction of algorithmic bias.

Full summary:

The motivation behind the paper is to highlight the key research directions in Fair ML that provide a scientific foundation for understanding algorithmic bias. These broadly include-  identifying bias encoded in data without access to outcomes (for example we have access to data about who was arrested and not who committed the crime), the utilitarian approach to optimization and how it caters purely to the majority without taking into account minority groups and the ethics of exploration. The role of exploration is a key one since in order to validate our predictions we must have data that enumerates how the outcome in fact played out. This brings up several important questions such as: Is the impact of exploration overwhelmingly felt by one subgroup? If we deem the risks of exploration too high, by how much does a lack of exploration slow learning? Is it ethical to sacrifice the well-being of current populations for the perceived well-being of future populations? 

The next important research direction is one that seeks to formalize the definition of Fairness. There are several proposed definitions, the most popular one being the statistical definition of Fairness. Such a formulation enforces parity in some chosen statistical measure across all groups in the data. The simplicity, assumption-free nature, and the ease with which a statically fair allocation can be verified makes this definition popular. However, a major shortcoming is the proven impossibility of simultaneously equalizing multiple desirable statistical measures. A statistical definition of fairness can also be computationally expensive to model. 

The second popular notion is that of Individual Fairness, which enforces that, for a given task, the algorithm treats individuals who are similar, similarly. While this is richer, semantically, it makes strong assumptions that are difficult to realize practically.  

Chouldechova and Roth then go on to present questions around Intersectional Fairness, namely: how different algorithmic biases compound for individuals who fall at the intersection of multiple protected groups. They also question the feasibility of a ā€˜good’ metric of fairness and whether such a notion will be accessible while making predictions, and the existence of an ā€˜agnostic’ notion of Fairness that does not rely on any one measure, but instead takes human feedback to correct for bias. 

Another important consideration is the dynamics of Fairness. Models are seldom deployed in one-shot settings and are usually used in conjunction with several other predictors. In such a setting, how does compositionality affect algorithmic fairness? ie. do individual components that satisfy conditions of ā€˜fairness’, continue to adhere to the same degree of fairness when composed together to decide a single outcome? 

Another source of dynamism is the impact that algorithmic decision-making systems have on the environment. Models that determine outcomes also influence the incentives of those who interact with them and hence it becomes imperative to consider long-term dynamics when designing ā€˜fair’ algorithms. We also need to reconcile the individual motives of the different actors in the system and incentivize them to behave ethically.

Lastly, Chouldechova and Roth enumerate open questions in modeling and correcting for bias in data, namely: How does bias arise in data? How do we correct for it? How do we take into account feedback loops, where biased predictions further lead to biased training data in future epochs? Enforcing any notion of fairness on biased data would see a drop in model accuracy and this begs the question of how we go about validating our ā€˜fair’ predictions.


Original paper by Alexandra Chouldechova and Aaron Roth: https://dl.acm.org/doi/pdf/10.1145/3376898

Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

šŸ” SEARCH

Spotlight

AI Policy Corner: New York City Local Law 144

Canada’s Minister of AI and Digital Innovation is a Historic First. Here’s What We Recommend.

Am I Literate? Redefining Literacy in the Age of Artificial Intelligence

AI Policy Corner: The Texas Responsible AI Governance Act

AI Policy Corner: Singapore’s National AI Strategy 2.0

related posts

  • More Trust, Less Eavesdropping in Conversational AI

    More Trust, Less Eavesdropping in Conversational AI

  • Value-based Fast and Slow AI Nudging

    Value-based Fast and Slow AI Nudging

  • Warning Signs: The Future of Privacy and Security in an Age of Machine Learning  (Research summary)

    Warning Signs: The Future of Privacy and Security in an Age of Machine Learning (Research summary)

  • South Korea as a Fourth Industrial Revolution Middle Power?

    South Korea as a Fourth Industrial Revolution Middle Power?

  • Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML

    Rethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML

  • Ethics-based auditing of automated decision-making systems: intervention points and policy implicati...

    Ethics-based auditing of automated decision-making systems: intervention points and policy implicati...

  • AI Bias in Healthcare: Using ImpactPro as a Case Study for Healthcare Practitioners’ Duties to Engag...

    AI Bias in Healthcare: Using ImpactPro as a Case Study for Healthcare Practitioners’ Duties to Engag...

  • Research summary: Decision Points in AI Governance

    Research summary: Decision Points in AI Governance

  • AI Framework for Healthy Built Environments

    AI Framework for Healthy Built Environments

  • Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback an...

    Humans are not Boltzmann Distributions: Challenges and Opportunities for Modelling Human Feedback an...

Partners

  • Ā 
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • Ā© MONTREAL AI ETHICS INSTITUTE. All rights reserved 2024.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.