• Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Core Principles of Responsible AI
    • Accountability
    • Fairness
    • Privacy
    • Safety and Security
    • Sustainability
    • Transparency
  • Special Topics
    • AI in Industry
    • Ethical Implications
    • Human-Centered Design
    • Regulatory Landscape
    • Technical Methods
  • Living Dictionary
  • State of AI Ethics
  • AI Ethics Brief
  • 🇫🇷
Montreal AI Ethics Institute

Montreal AI Ethics Institute

Democratizing AI ethics literacy

The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks (Research Summary)

November 30, 2020

Summary contributed by our researcher Erick Galinkin (@ErickGalinkin), who’s also Principal AI Researcher at Rapid7.

*Link to original paper + authors at the bottom.


Overview: Neural networks have shown amazing ability to learn on a variety of tasks, and this sometimes leads to unintended memorization. This paper explores how generative adversarial networks may be used to recover some of these memorized examples.


Model inversion attacks are a type of attack which abuse access to a model by attempting to infer information about the training data set. Effective model inversion attacks have largely been on extremely simple models such as linear regression and logistic regression, showing little promise in deep neural networks. However, generative adversarial networks (GANs) provide the ability to approximate these data sets.

Using techniques similar to image inpainting for obscured or damaged images, the GAN creates semantically plausible pixels based on what has been inferred about the sensitive features in the training data. A Wasserstein-GAN is used to set up a min-max problem as the loss function, and some auxiliary knowledge about the private images are provided to the attacker. This serves as an additional input to the generator. The generator then passes the recovered images to both the target network and a discriminator. The loss from both of these inferences is combined to optimize the generator. 

Using facial recognition classifiers as a model, Zhang et al. find that generative model inversion is significantly more effective than existing model inversion methods. Notably, more powerful models which have more layers and parameters are more susceptible to the attack.

Zhang et al. also find that pre-training the GAN on auxiliary data from the training distribution helps recovery of private data significantly. However, even training on similar data with a different distribution – such as pre-training on the PubFig83 dataset and attacking a model trained on the CelebA dataset still outperforms existing model inversion attacks by a large margin. Some image pre-processing can further improve the accuracy of the GAN in generating target data.

Finally, Zhang et al. investigated the implications of differential privacy in recovering images. They note that differentially private facial recognition models are very difficult to produce with acceptable accuracy in the first place, due to the complexity of the task. Thus, using MNIST as a reference dataset, they find that generative model inversion can expose private information from differentially private models even with strong privacy guarantees, and the strictness of the guarantee does not impact the ability to recover data. They suggest that this is likely because “DP, in its canonical form, only hides the presence of a single instance in the training set; it does not explicitly aim to protect attribute privacy.”


Original paper by Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Dawn Song: https://arxiv.org/abs/1911.07135

Want quick summaries of the latest research & reporting in AI ethics delivered to your inbox? Subscribe to the AI Ethics Brief. We publish bi-weekly.

Primary Sidebar

🔍 SEARCH

Spotlight

ALL IN Conference 2025: Four Key Takeaways from Montreal

Beyond Dependency: The Hidden Risk of Social Comparison in Chatbot Companionship

AI Policy Corner: Restriction vs. Regulation: Comparing State Approaches to AI Mental Health Legislation

Beyond Consultation: Building Inclusive AI Governance for Canada’s Democratic Future

AI Policy Corner: U.S. Executive Order on Advancing AI Education for American Youth

related posts

  • It doesn't tell me anything about how my data is used'': User Perceptions of Data Collection Purpos...

    "It doesn't tell me anything about how my data is used'': User Perceptions of Data Collection Purpos...

  • Theorizing Femininity in AI: a Framework for Undoing Technology’s Gender Troubles (Research Summary)

    Theorizing Femininity in AI: a Framework for Undoing Technology’s Gender Troubles (Research Summary)

  • A roadmap toward empowering the labor force behind AI

    A roadmap toward empowering the labor force behind AI

  • Dual Governance: The intersection of centralized regulation and crowdsourced safety mechanisms for G...

    Dual Governance: The intersection of centralized regulation and crowdsourced safety mechanisms for G...

  • Experts Doubt Ethical AI Design Will Be Broadly Adopted as the Norm Within the Next Decade

    Experts Doubt Ethical AI Design Will Be Broadly Adopted as the Norm Within the Next Decade

  • From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting

    From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting

  • Achieving Fairness at No Utility Cost via Data Reweighing with Influence

    Achieving Fairness at No Utility Cost via Data Reweighing with Influence

  • Cinderella’s shoe won’t fit Soundarya: An audit of facial processing tools on Indian faces

    Cinderella’s shoe won’t fit Soundarya: An audit of facial processing tools on Indian faces

  • Deepfakes and Domestic Violence: Perpetrating Intimate Partner Abuse Using Video Technology

    Deepfakes and Domestic Violence: Perpetrating Intimate Partner Abuse Using Video Technology

  • ABScribe: Rapid Exploration of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large ...

    ABScribe: Rapid Exploration of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large ...

Partners

  •  
    U.S. Artificial Intelligence Safety Institute Consortium (AISIC) at NIST

  • Partnership on AI

  • The LF AI & Data Foundation

  • The AI Alliance

Footer

Categories


• Blog
• Research Summaries
• Columns
• Core Principles of Responsible AI
• Special Topics

Signature Content


• The State Of AI Ethics

• The Living Dictionary

• The AI Ethics Brief

Learn More


• About

• Open Access Policy

• Contributions Policy

• Editorial Stance on AI Tools

• Press

• Donate

• Contact

The AI Ethics Brief (bi-weekly newsletter)

About Us


Founded in 2018, the Montreal AI Ethics Institute (MAIEI) is an international non-profit organization equipping citizens concerned about artificial intelligence and its impact on society to take action.


Archive

  • © 2025 MONTREAL AI ETHICS INSTITUTE.
  • This work is licensed under a Creative Commons Attribution 4.0 International License.
  • Learn more about our open access policy here.
  • Creative Commons License

    Save hours of work and stay on top of Responsible AI research and reporting with our bi-weekly email newsletter.