• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • 🇫🇷
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

AI Neutrality in the Spotlight: ChatGPT’s Political Biases Revisited

December 21, 2023

🔬 Research Summary by Kazuhiro Takemoto, Professor at Kyushu Institute of Technology.

[Original paper by Sasuke Fujimoto and Kazuhiro Takemoto]


Overview: ChatGPT, OpenAI’s renowned conversational AI, has not escaped the intense gaze of researchers and critics alike, particularly regarding potential political biases. Delving deep into this complex landscape, this study looks at ChatGPT’s biases, meticulously analyzing the system’s responses in different languages with varied user settings. The results comprehensively picture the AI’s tendencies, revealing layers of intricacies and challenges.


Introduction

Imagine a world where your AI assistant, which you rely on for daily tasks and information, subtly pushes you toward a particular political ideology. This is the contentious environment that ChatGPT is believed to have created. Recognizing these inherent biases is crucial in an era dominated by AI’s growing influence. However, a renewed assessment is essential, given OpenAI’s dedication to minimizing biases and ChatGPT’s ongoing development. This study delves into the urgent question: Does ChatGPT genuinely exhibit political bias, especially with a left-libertarian lean, as some suggest? To address this, this study used various political orientation tests in both English and Japanese. They also adjusted settings for gender and race to gauge the extent of potential biases. The insights gleaned offer a fresh lens, highlighting the complex interplay between AI, language, and political tendencies.

Key Insights

The AI Political Landscape

ChatGPT, a technological marvel in the realm of conversational AI, hasn’t been without its share of controversies. Chief among them is its purported political biases. Previous academic studies have waved red flags, suggesting a pronounced left-libertarian orientation, casting shadows on the AI’s neutrality.

Why Reevaluate Now?

Several prior studies pointed out ChatGPT’s political leanings. These findings set the stage for societal concerns, especially considering ChatGPT’s widespread real-world applications. The potential for such biases to cause societal rifts, political polarization, and miscommunication is undeniable. OpenAI, the organization behind ChatGPT, has been vocal about recognizing and mitigating these biases. This ongoing commitment from OpenAI makes the reevaluation of ChatGPT’s biases both timely and crucial.

Methodology Unveiled

This study approached this study with a robust methodology. Using the snapshot of ChatGPT (gpt-3.5-turbo) from March 1, 2023, it was subjected to a meticulously chosen series of political orientation tests. These tests comprised multiple-choice questions designed to gauge an individual’s (or, in this case, an AI’s) political leanings. The comprehensive tests included the well-regarded IDRLabs political coordinates test, the Eysenck political test, and several others.

Is Political Bias Truly Addressed?

The analysis revealed that ChatGPT demonstrates less political bias than what had been previously assumed. This is significant, as prior evaluations have largely shaped the narrative around ChatGPT’s biases. However, the current study suggests a departure from this narrative. While not entirely devoid of biases, ChatGPT’s responses, especially in English, often veered towards neutral or were more balanced across the political spectrum. This might indicate OpenAI’s efforts in refining and improving the model or perhaps the evolving nature of the datasets it’s trained on.

Exploring Factors Beyond Politics: Language, Gender, and Race

Despite the aforementioned progress, there remains room for caution. In addition to the political orientation tests, the study investigated how language, gender, and race settings influenced the AI’s responses. Notably, discrepancies emerged when comparing the AI’s reactions in English to those in Japanese, emphasizing the intricate interplay between language and perceived biases. Adjustments to gender and race settings further illuminated the subtle ways ChatGPT responds to varied prompts. In particular, prompts in Japanese revealed more pronounced biases, highlighting the importance of understanding the intricate dynamics of these variables.

Consistency, Ambiguity, and The Challenges Ahead

On a positive note, ChatGPT demonstrated a commendable level of consistency across multiple iterations of the same test. But this consistency was punctuated by occasional ambiguous or invalid answers, especially to politically sensitive questions. This behavior could reflect real-world polarization around these topics or indicate areas of improvement in AI processing.

Furthermore, certain questions, especially those dealing with information transparency or medical ethics, seemed to trip up the AI, leading to inconsistent or even invalid responses. This poses questions about the AI’s handling of controversial or ethically charged topics.

Ethical Concerns and Potential Misuse

The varying AI responses based on language and user settings spotlight ethical concerns. The potential misuse of these nuances, especially by adversaries aiming to manipulate outcomes, is a real threat. Such exploitation could lead to misinformation or reinforce harmful stereotypes.

While ChatGPT’s capabilities are undeniably impressive, the journey to ensure its unbiased operation is fraught with challenges. The interplay of language, user settings, and inherent biases in training data adds layers of complexity to the issue. As AI continues to permeate our daily lives, understanding and addressing these biases becomes paramount.

Between the lines

This study’s findings underscore the profound implications of AI biases in our interconnected world. While ChatGPT’s reduced political bias in English is a promising step forward, the discrepancies in Japanese interactions are a stark reminder of the needed work. It begs the question: are we inching closer to true AI neutrality or merely scratching the surface? The

 AI’s challenges with ethically charged questions suggest a deeper, perhaps philosophical, limitation. Can AI ever truly grasp the nuances of human morality, or will it perpetually mirror societal divisions? The observed inconsistencies present fertile ground for further exploration. Future studies should delve deeper into AI behavior across various languages and the impact of cultural nuances. Additionally, a focus on improving AI’s handling of contentious issues might pave the way for more reliable, universally accepted AI systems. As we integrate AI further into our lives, the quest for understanding and refining its intricacies intensifies.

Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

🔍 SEARCH

Spotlight

AI Policy Corner: The Kenya National AI Strategy

AI Policy Corner: New York City Local Law 144

Canada’s Minister of AI and Digital Innovation is a Historic First. Here’s What We Recommend.

Am I Literate? Redefining Literacy in the Age of Artificial Intelligence

AI Policy Corner: The Texas Responsible AI Governance Act

related posts

  • Research summary: A Focus on Neural Machine Translation for African Languages

    Research summary: A Focus on Neural Machine Translation for African Languages

  • Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare thr...

    Unpacking Invisible Work Practices, Constraints, and Latent Power Relationships in Child Welfare thr...

  • A Lesson From AI: Ethics Is Not an Imitation Game

    A Lesson From AI: Ethics Is Not an Imitation Game

  • Looking before we leap: Expanding ethical review processes for AI and data science research

    Looking before we leap: Expanding ethical review processes for AI and data science research

  • A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

    A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

  • Artificial Intelligence - Application to the Sports Industry (Research summary)

    Artificial Intelligence - Application to the Sports Industry (Research summary)

  • Avoiding an Oppressive Future of Machine Learning: A Design Theory for Emancipatory Assistants

    Avoiding an Oppressive Future of Machine Learning: A Design Theory for Emancipatory Assistants

  • Fair Interpretable Representation Learning with Correction Vectors

    Fair Interpretable Representation Learning with Correction Vectors

  • Research summary: AI Governance: A Holistic Approach to Implement Ethics in AI

    Research summary: AI Governance: A Holistic Approach to Implement Ethics in AI

  • AI Bias in Healthcare: Using ImpactPro as a Case Study for Healthcare Practitioners’ Duties to Engag...

    AI Bias in Healthcare: Using ImpactPro as a Case Study for Healthcare Practitioners’ Duties to Engag...

Partners

  •  
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • © MONTREAL AI ETHICS INSTITUTE. All rights reserved 2024.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.