• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • 🇫🇷
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

Transparency as design publicity: explaining and justifying inscrutable algorithms

November 5, 2021

🔬 Research summary by Dr. Marianna Ganapini (@MariannaBergama), our Faculty Director.

[Original paper by Michele Loi, Andrea Ferrario, and Eleonora Viganòngsma]


Overview: It is often said that trustworthy AI requires systems to be transparent and/or explainable. The goal is to make sure that these systems are epistemically and ethically reliable, while also giving people the chance to understand the outcomes of those systems and the decisions made based on those outcomes. In this paper, the solution proposed stems from the relationship between “design explanations” and transparency: if we have access to the goals, the values and the built-in priorities of an algorithm system, we will be in a better position to evaluate its outcomes.

Introduction

How can we make AI more understandable? According to the authors of the paper, we care about making AI more intelligible mostly because we want to understand the normative reasons behind a certain AI prediction or outcome. In other words, we want to know: what justifies the outcome of a certain algorithmic assessment, why should I trust that outcome to act and form beliefs based on it? In the paper, the solution proposed stems from the relationship between “design explanations” and transparency: if we have access to the goals, the values and the built-in priorities of an algorithm-system, we will be in a better position to evaluate its outcomes.

Key Insights

The starting point for talking about transparency and explainability in AI is Lipton’s (2018) claim that interpretations of ML models are divided in two categories: model-transparency and post-hoc explanations. Post-hoc explanations look at the prediction of  a model and include, most prominently, counterfactual explanations (Wachter et al. 2017). These are based on certain “model features” which, if altered, change the outcome of the model, other things being equal. By looking at the features that impacted a certain outcome, one can in theory determine the (counterfactual) causes that produced that outcome. Though these tools are often used in explainable-AI, the authors of the paper are skeptical: they believe counterfactual explanations do not provide the necessary insights to understand the normative aspects of the model.  

Transparency should somehow tell us how the model works, at least in Lipton’s definition. However, the authors of the paper have something slightly different in mind: they believe transparency is really the result of making “design explanations” explicit. That is, we need to know what the system’s function is and how the system was designed to achieve that function. As the authors put it, “explaining the purpose of an algorithm requires giving information on various elements: the goal that the algorithm pursues, the mathematical constructs into which the goal is translated in order to be implemented in the algorithm, and the tests and the data with which the performance of the algorithm was verified.” 

Parallely, they see “design transparency of an algorithmic system to be the adequate communication of the essential information necessary to provide a satisfactory design explanation of such a system.” The most prominent type of transparency in this context is value transparency: we need an accessible account of what values were designed in the system, how they were implemented and to what extent (what tradeoffs were made). Embedded values are values that are designed as part of an algorithmic system and that the system is also able to show in its output. As the authors explain, “[o]nce the criteria to measure the degree of goal achievement are specified” the “a design explanation of an algorithm should provide information on the effective achievement of such objectives in the environment for which the system was built.” That is called “performance transparency” in the paper.

This approach is meant to shed light on the goals algorithmic systems are designed to achieve, the values and tradeoffs built into the systems, the set of priorities the system is designed to have and the benchmarks for evaluating success and failure of this design. The goal of transparency is ultimately to provide “the public with the essential elements that are needed in order to assess the justification […]  of the decisions” that are based on automated evaluations.  If the decisions are based on a system not designed – either intentionally or at the level of how the values are translated – to foster some ethical values, then one might reasonably suspect the decisions made won’t match some ethical requirements. More importantly, these decisions cannot be morally acceptable since they are not motivated by the right set of priorities. Understanding all this is a key requirement for evaluating AI and the decisions made based on its recommendations.

Between The Lines

In this very interesting paper, the authors offer some actionable recommendations for how to make AI more understandable which seem fully in line with the idea of achieving an “ethics by design” approach to AI. Yet, we also believe that counterfactual and post-hoc explanations could be part of this approach with the goal, for instance, of checking for things that might have gone wrong. Therefore, we would not exclude them from an account of explainability in AI and we recommend a comprehensive approach to make AI understandable to humans.

Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

🔍 SEARCH

Spotlight

Beyond Consultation: Building Inclusive AI Governance for Canada’s Democratic Future

AI Policy Corner: U.S. Executive Order on Advancing AI Education for American Youth

AI Policy Corner: U.S. Copyright Guidance on Works Created with AI

AI Policy Corner: AI for Good Summit 2025

AI Policy Corner: Japan’s AI Promotion Act

related posts

  • Representation Engineering: A Top-Down Approach to AI Transparency

    Representation Engineering: A Top-Down Approach to AI Transparency

  • Down the Toxicity Rabbit Hole: Investigating PaLM 2 Guardrails

    Down the Toxicity Rabbit Hole: Investigating PaLM 2 Guardrails

  • Mapping the Design Space of Human-AI Interaction in Text Summarization

    Mapping the Design Space of Human-AI Interaction in Text Summarization

  • Research summary: Different Intelligibility for Different Folks

    Research summary: Different Intelligibility for Different Folks

  • Target specification bias, counterfactual prediction, and algorithmic fairness in healthcare

    Target specification bias, counterfactual prediction, and algorithmic fairness in healthcare

  • Designing Fiduciary Artificial Intelligence

    Designing Fiduciary Artificial Intelligence

  • Fairness Uncertainty Quantification: How certain are you that the model is fair?

    Fairness Uncertainty Quantification: How certain are you that the model is fair?

  • Artificial Intelligence and Inequality in the Middle East: The Political Economy of Inclusion

    Artificial Intelligence and Inequality in the Middle East: The Political Economy of Inclusion

  • Research summary: Learning to Diversify from Human Judgments - Research Directions and Open Challeng...

    Research summary: Learning to Diversify from Human Judgments - Research Directions and Open Challeng...

  • Combatting Anti-Blackness in the AI Community

    Combatting Anti-Blackness in the AI Community

Partners

  •  
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • © 2025 MONTREAL AI ETHICS INSTITUTE.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.