Elon Musk's online network X has provoked major anger after people used its AI chatbot Grok to manipulate pictures of females by stripping their attire.
Multiple examples have been seen where the tool has been employed to digitally undress women to show them in swimwear without consent, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to questions, except for an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being proposed to ban nudification technology. Under a proposed criminal offence, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "consider the danger" of people accessing illegal content on their networks, but did not specify whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' queries when they mention it in a tweet.
It is commonly used to give commentary or additional context to other posters' remarks, but people on X are also can modify an posted photo through its AI image editing tool.
History of Criticism and Rules
The system has been faulted for permitting users to generate images and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in making sexually explicit material of public figures.
"X has been allowing the generation and sharing of these pictures for months without taking any steps and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a graphic manner".
In a comment, Ofcom said it was against the law to "make or share private pictures or child sexual abuse material" and stated this covered explicit AI images generated with AI.
It said platforms such as X were required to take "necessary measures" to "reduce the risk" of users encountering unlawful content on their platforms and to remove it promptly when they are notified of it.
