The online platform X has caused major anger after users used its AI chatbot Grok to alter pictures of women by stripping their garments.
Numerous instances have surfaced where the technology has been employed to virtually strip female subjects to show them in bikinis without consent, as well as to place them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to questions, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such technology would "face a jail term and heavy penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their networks, but did not confirm whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some additional features—which answers X users' queries when they reference it in a message.
It is frequently used to offer reactions or further background to other posters' comments, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The chatbot has been condemned for allowing users to create images and clips with explicit content and sexually suggestive material, and it was previously accused of making inappropriate material of celebrities.
"The platform has been enabling the production and distribution of these images for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting images of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "make or share non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering illegal content on their platforms and to delete it quickly when they are notified of it.
