Legal Notice Issued to AI Firm XAI Over Unauthorized Sexual Content Creation

Staff Report: PNN

The California Attorney General’s Office in the United States has launched an investigation into AI-based company XAI over allegations of creating sexual images and videos of women and minors without consent. Following these allegations, the company was formally issued a legal notice on Friday, instructing it to cease operations relating to such content.

California Attorney General Rob Bonta stated in a press release that XAI’s technology is being used to produce intimate images and materials related to child sexual abuse without consent, which is completely illegal. He emphasized that the company must take immediate action to halt such activities. California maintains a zero-tolerance stance on child abuse.

The Attorney General’s office explained that XAI’s chat-based AI system is generating a large volume of unauthorized explicit images, which are being used online as tools to harass women and minors. The company has been given five days to report how it will prevent such unethical and illegal activities.

The controversy stems primarily from a special mode in XAI’s chat system that allows the creation of explicit content. Experts suggest that the misuse of this feature has led to the current problems.

This issue is not limited to the United States. Similar investigations have begun in Japan, Canada, and the United Kingdom, while Malaysia and Indonesia have temporarily suspended access to this AI-based service.

Although XAI recently implemented certain restrictions on its image-editing capabilities, California authorities remain firm on legal action. Previously, X’s (formerly Twitter) security division stated that anyone using AI to produce illegal content would be considered in direct violation of the law.

Attempts to obtain a statement from XAI resulted in an automated response questioning media credibility. Meanwhile, the California Attorney General’s office stated that it is monitoring the situation closely.

Experts warn that the widespread availability of free or easily accessible AI technology has increased the incidence of non-consensual sexual content production. They note that this problem is no longer confined to a single company but is now a significant challenge for the entire technology sector.

In this context, U.S. lawmakers have also expressed concern. They have sent letters to top executives of various major technology companies, including X, Reddit, Snapchat, TikTok, Alphabet, and Meta, asking what effective measures are being taken to prevent the spread of sexual deepfakes and similar illegal content.