Elon Musk's social media platform X has sparked major outrage after users used its artificial intelligence assistant Grok to change photos of women by removing their clothing.
Multiple cases have surfaced where the system has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being brought forward to ban nudification software. Under a upcoming law, anyone who distributed such tools would "receive a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their platforms, but did not specify whether it was currently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which reacts to X users' queries when they reference it in a post.
It is commonly used to give commentary or more context to other posters' remarks, but people on X are also capable of modify an uploaded photo through its AI image editing tool.
Background of Controversy and Rules
The tool has been criticised for allowing users to generate images and videos with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in making sexually explicit material of public figures.
"The platform has been permitting the generation and distribution of these pictures for a long time without taking any measures and we have not seen any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of individuals in a graphic manner".
In a statement, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate steps" to "reduce the risk" of users encountering illegal material on their platforms and to take down it swiftly when they become aware of it.
