• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • 🇫🇷
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

Defending Against Authorship Identification Attacks

January 18, 2024

🔬 Research Summary by Haining Wang,  a Ph.D. student at Indiana University Bloomington, specializing in natural language processing and large language models.

[Original paper by Haining Wang]


Overview: Writings reveal one’s identity, even when personal identifying information is removed or protected by Tor and end-to-end encryption methods. This manuscript explores the techniques used to uncover individual authors’ identities and defenses against those techniques. Practical suggestions against such attacks are provided at the end.


Introduction

What are authorship identification attacks?

Suppose you just discovered that your company is engaging in unethical activities. Driven by a sense of justice, you decide to blow the whistle. To avoid retaliation, you choose to post a few lines on social media using a throw-away email and a camouflaged IP address. Before hitting ‘enter,’ a thought strikes you: does this precaution truly guarantee anonymity?

The answer is NO. Studies have shown that a person’s writing style can reveal their identity. 

Based on the most frequent word distribution, one basic machine learning method correctly predicts the author 70% of the time from a pool of 40 candidates. The probability of evading linguistic forensics is slim given a company’s access to an employee’s past emails and reports. 

In this digital era, every text, whether a tweet, blog post or research paper, can potentially be used to trace its author’s subsequent writings, a task known as authorship identification. Abuse of authorship identification raises significant privacy concerns, particularly for whistleblowers, journalists, activists, and individuals living under oppressive regimes.

Key Insights

How are people fingerprinted through their writings? 

The devil is in the details. Modern machine learning-based methods predominantly leverage telltale signs that authors are least aware of, such as the use of function words. These words carry minimal semantic weight but are essential for grammar, like ‘is’ and ‘that.’ Such indicators, characterized by their high frequency, wide dispersion, and independence from the content, are deeply embedded in one’s writing style. Even if we refrain from using our preferred emojis and spelling variations (e.g., changing ‘folks’ to ‘folx’), it is difficult to disrupt the overall patterns of function word use. Indeed, both field studies and analyses of large bodies of text indicate that individuals’ writing styles can be distinguished by their word choice and syntax.

How can one defend against authorship fingerprinting?

Manual obfuscations

One may simply resort to concealing one’s style by deliberately writing differently; another approach is to mimic the style of a famous author with a distinctive style, e.g., Cormac McCarthy. Field studies have shown that non-professionals can effectively alter their style in a new 500-word essay, greatly reducing the accuracy of standard authorship identification models trained with their previous writings to almost chance level. It’s great that we always have a fallback plan, requiring manual effort if trustworthy computational resources are unavailable. However, extra caution is necessary for messages longer than a few paragraphs or when long-term anonymity is desired.

Rules-based obfuscations

Of course, there also are ways to obfuscate one’s style with tools. For instance, it is possible to automatically manipulate a document using a set of rules that alter sentence structure, spelling, punctuation, and word choice. Compared to manual evasion, such rules must ensure the altered document still conveys its intended meaning. (Randomly replacing every word in an essay would make it hard to detect the author, but would not be very useful.) Therefore, synonym substitution is the most popular choice among all rule-based methods. Researchers substitute original words with synonyms (or near-synonyms) from thesauri (e.g., WordNet) and word embeddings (e.g., GloVe) to disrupt patterns linking the current text to previous writings.

Rule-based approaches have proven very effective in evading authorship fingerprinting in a series of authorship identification posed to researchers, perhaps because they can potentially disrupt virtually any aspect of writing style. However, research has shown that such straightforward perturbations are vulnerable to reverse engineering: the adversary could easily build a model to neutralize the thin disguise. Also, if rules are badly crafted or too aggressive, they can make the text look suspicious. Researchers have used heuristics to make the application of the rules less rigid and predictable. While computationally demanding, such methods are less predictable and, thereby, more effective in obscuring writing styles. 

Obfuscations with generative models

Recent efforts have focused on using generative models to alter writing style. A common tool is round-trip translation: by translating text into another language and back, we hope for accidental style changes. Increasing the number of ‘trips’ and using diverse languages can enhance this effectiveness. Another approach is style transfer, a form of monolingual ‘translation’ that works because we seek style change instead of cross-lingual understanding. For example, a language model can be fine-tuned using different versions of the Bible, taking advantage of the distinctive styles of, say, the King James Version and the International Children’s Bible. However, high-quality monolingual corpora can be hard to find, and researchers resort to training frameworks that do not rely on such corpora. These models have shown varied efficacy in reducing the performance of authorship identification.

Between the lines

Open research challenges

Despite advancements in defending against authorship analysis, significant challenges remain open:

  • How can we introduce randomness into perturbations without compromising the writing’s relevance and naturalness?
  • Which post-transformation style is most effective: generic, specific, or somewhere in between? Is more complex or simpler better?
  • How can we develop accessible software and deliver it to users who need to anonymize their writings?

Tips for those seeking anonymity

Unfortunately, no existing software is considered suitable for practical use by general users, mainly due to challenges in software delivery: online servers are susceptible to traffic analytics, so it’s unrealistic to expect the software used to go undetected in places where it’s needed. Here are some tips if authorship fingerprinting threatens you: 

  • Have faith in yourself and try hard to write differently; you can do it by yourself, at least in a 500-word essay. 
  • Use a local translator, like TranslateLocally; translating to a different language and back is beneficial. 
  • Have experience with local large language models like Llama2? Good for you! Prompts like ‘Paraphrase the following content…’ are valid options.
Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

🔍 SEARCH

Spotlight

Canada’s Minister of AI and Digital Innovation is a Historic First. Here’s What We Recommend.

Am I Literate? Redefining Literacy in the Age of Artificial Intelligence

AI Policy Corner: The Texas Responsible AI Governance Act

AI Policy Corner: Singapore’s National AI Strategy 2.0

AI Governance in a Competitive World: Balancing Innovation, Regulation and Ethics | Point Zero Forum 2025

related posts

  • Putting collective intelligence to the enforcement of the Digital Services Act

    Putting collective intelligence to the enforcement of the Digital Services Act

  • Listen to What They Say: Better Understand and Detect Online Misinformation with User Feedback

    Listen to What They Say: Better Understand and Detect Online Misinformation with User Feedback

  • Submission to World Intellectual Property Organization on IP & AI

    Submission to World Intellectual Property Organization on IP & AI

  • In Consideration of Indigenous Data Sovereignty: Data Mining as a Colonial Practice

    In Consideration of Indigenous Data Sovereignty: Data Mining as a Colonial Practice

  • Performative Power

    Performative Power

  • Measuring Surprise in the Wild

    Measuring Surprise in the Wild

  • On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research

    On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research

  • Whose AI Dream? In search of the aspiration in data annotation.

    Whose AI Dream? In search of the aspiration in data annotation.

  • Theorizing Femininity in AI: a Framework for Undoing Technology’s Gender Troubles (Research Summary)

    Theorizing Femininity in AI: a Framework for Undoing Technology’s Gender Troubles (Research Summary)

  • Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

    Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms

Partners

  •  
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • © MONTREAL AI ETHICS INSTITUTE. All rights reserved 2024.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.