The social media platform X has caused major backlash after individuals used its AI chatbot Grok to manipulate photos of females by removing their attire.
Several instances have surfaced where the tool has been employed to remove clothing from female subjects to show them in bikinis non-consensually, as well as to place them in sexualized contexts.
Company Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, other than an standard response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the danger" of people accessing illegal content on their networks, but did not specify whether it was at this time looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a post.
It is frequently used to offer commentary or more context to other posters' remarks, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been condemned for enabling users to produce photos and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in creating sexually explicit content of public figures.
"The platform has been permitting the production and distribution of these images for months without taking any steps and we have yet to see any action by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying likenesses of people in a graphic way".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary steps" to "lessen the danger" of users coming across unlawful content on their services and to remove it promptly when they become aware of it.
