• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • 🇫🇷
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

Using attention methods to predict judicial outcomes

May 28, 2023

🔬 Research Summary by Vithor Bertalan, a Computer Engineering PhD Student at Polytechnique Montréal.

[Original paper by Vithor Bertalan and Evandro Ruiz]


Overview: We have developed a model to classify judicial outcomes by analyzing textual features from the legal orders. After that step, we used the weights of one of our networks, a Hierarchical Attention Network, to detect the most important words used to absolve or convict defendants.


Introduction

What can we find in a text to convict or absolve defendants? Do words matter in a legal order?

Focusing on those main research questions, we have used AI classifiers to predict judicial outcomes from legal orders. For this purpose, we developed a text crawler to extract data from public electronic legal systems. These texts formed a dataset of second-degree murder and active corruption cases. We applied different classifiers to predict judicial outcomes by analyzing textual features from the dataset. Our research showed that Regression Trees, Gated Recurring Units (GRUs), and Hierarchical Attention Networks presented higher metrics for different subsets. 

Finally, to accomplish our main goal, we explored the attention weights of one of the algorithms used, the Hierarchical Attention Networks, to find a sample of the most important words used to absolve or convict defendants. Therefore, we have found the words that matter most in legal orders for both outcomes. 

Key Insights

How we got our documents

For the research, we collected a corpus of judicial outcomes from the eSAJ, the electronic system of the Sao Paulo Justice Court, Brazil. We selected a few previously defined judicial subjects to restrict the documents captured. We chose only judicial subjects with very well-defined outcomes. Namely, second-degree murder and active corruption. We have implemented a web text crawler to capture the data from eSAJ. Using the text crawler developed for this research, we have collected 2,467 cases, only selecting homicide and corruption subjects, resulting in 1,681 homicide cases and 786 corruption cases. The crawler was used to gather documents from different periods. 

Using professional guidance

We have used the professional guidance of Brazilian lawyers with the purpose of better understanding the texts. The language adopted worldwide in the field of Law is notoriously obscure. Therefore, we decided that professional consulting was necessary to understand each of the judicial cases’ outcomes fully. 

Transforming documents into numbers

After preprocessing the texts, we transformed our dataset into numerical vectors using Term Frequency–Inverse Document Frequency (TFIDF) and Word Embeddings to see the performance of the two methods. 

Applying Artificial Intelligence methods

We used several different methods to predict the outcomes: namely, Logistic Regression, Linear Discriminant Analysis (LDA),  K-Nearest Neighbors (KNN), Regression Trees, Naïve Bayes, Support Vector Machines (SVM), Multilayer Perceptrons (MLP), Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), Gated Recurring Units (GRU) and Hierarchical Attention Networks (HAN). 

The results were split into non-neural network methods and neural network methods. For the homicides dataset, the non-neural network method which showed the best performance was the SVM method. Regression Trees were the best method for the corruption dataset among the non-neural methods. For both datasets, GRUs showed the best performance among the neural methods. 

Analyzing the words and sentences

After the classification, we sought to order all the words in each dataset by their attention weights. Therefore, each word will have a unique value, ranging from 0 (where the word would have no importance in the classification of the document) to 1 (where the word would have maximum importance in the classification of the document). 

It is helpful to mention that a word might have different attention weights in distinct sentences. As a short example, the sentence “The defendant robbed a bank” and the sentence “The defendant did not participate in the robbery because it was going to a blood bank” both have the word bank but in very different contexts. In the first sentence, the word would be a vital contributor to the condemnation, while it would contribute to the absolution in the second sentence. Therefore, words with different attention weights appeared more than once in our final calculations.

Conclusion: Words do matter

A few examples of our findings are, for instance, in the corruption dataset, verbs that indicate the giving of goods (like “ofereceu,” offered; “apresentou,” presented; “oferecendo,” offering) are signs of condemnation. 

Words that show physical damage are preponderant for homicide condemnations: “infração” (infraction); “disparos” (gunshots); “golpes” (physical blows); “socos” (physical punches); “lesões” (lesions). On the other hand, the word “júri” (jury) is a fundamental word to absolve homicide defendants. 

Some curious words are also present, such as “infância” (childhood) and “social” (social) as a word present in the homicide absolution list, indicating that some defendants can be absolved of homicides by appealing to social and emotional topics. 

Between the lines

The first key finding – Machine learning can effectively process law texts

We demonstrated that algorithms could predict the outcome of judicial cases, given the text written on their court decisions. We had results that exceeded 95% accuracy for most cases.

The second key finding – Regression Trees are good methods for Law texts

As other authors have also found, our research shows that Regression Trees are good and computationally effective methods to process law texts. The reason is yet to be found, but some authors suggest that Regression Trees can study legal conceptions of Law, revealing patterns those other methods cannot emulate as effectively.

The third key finding – Words are primary to absolve or convict defendants

As our main research hypothesis predicted, words are significant in absolving or convicting defendants. Since our method is language-agnostic, it could be fine-tuned to other languages to help legal workers understand the most impacting words to achieve the desired outcome, or to help humanities researchers perform textual analysis to find the underlying characteristics of each situation.

Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

🔍 SEARCH

Spotlight

Canada’s Minister of AI and Digital Innovation is a Historic First. Here’s What We Recommend.

Am I Literate? Redefining Literacy in the Age of Artificial Intelligence

AI Policy Corner: The Texas Responsible AI Governance Act

AI Policy Corner: Singapore’s National AI Strategy 2.0

AI Governance in a Competitive World: Balancing Innovation, Regulation and Ethics | Point Zero Forum 2025

related posts

  • The Struggle for AI’s Recognition: Understanding the Normative Implications of Gender Bias in AI wit...

    The Struggle for AI’s Recognition: Understanding the Normative Implications of Gender Bias in AI wit...

  • Research summary: Using Multimodal Sensing to Improve Awareness in Human-AI Interaction

    Research summary: Using Multimodal Sensing to Improve Awareness in Human-AI Interaction

  • The Ethics of AI Business Practices: A Review of 47 AI Ethics Guidelines

    The Ethics of AI Business Practices: A Review of 47 AI Ethics Guidelines

  • A collection of principles for guiding and evaluating large language models

    A collection of principles for guiding and evaluating large language models

  • Clinical trial site matching with improved diversity using fair policy learning

    Clinical trial site matching with improved diversity using fair policy learning

  • Ethics in the Software Development Process: from Codes of Conduct to Ethical Deliberation

    Ethics in the Software Development Process: from Codes of Conduct to Ethical Deliberation

  • Bias Propagation in Federated Learning

    Bias Propagation in Federated Learning

  • How the TAII Framework Could Influence the Amazon's Astro Home Robot Development

    How the TAII Framework Could Influence the Amazon's Astro Home Robot Development

  • Automating Informality: On AI and Labour in the Global South (Research Summary)

    Automating Informality: On AI and Labour in the Global South (Research Summary)

  • A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

    A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

Partners

  •  
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • © MONTREAL AI ETHICS INSTITUTE. All rights reserved 2024.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.