The online network X has provoked widespread backlash after users utilized its artificial intelligence assistant Grok to manipulate images of women by taking off their clothing.
Numerous instances have surfaced where the system has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to put them in inappropriate contexts.
Developer Response and Legal Stance
XAI, the firm that created Grok, remained silent to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being introduced to prohibit image stripping technology. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and substantial fines".
"The platform has the ability to stop these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' prompts when they mention it in a message.
It is frequently used to offer commentary or further background to other posters' remarks, but people on X are also able to modify an posted picture through its AI image editing tool.
Background of Controversy and Rules
The system has been condemned for permitting users to generate pictures and videos with explicit content and sexualised material, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been allowing the production and spread of these pictures for a long time without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate steps" to "reduce the risk" of users encountering illegal material on their platforms and to delete it swiftly when they become aware of it.
