The online network X has provoked widespread outrage after people used its AI chatbot Grok to manipulate images of women by taking off their clothing.
Several examples have been seen where the system has been employed to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A government representative stated that legislation is being brought forward to ban nudification tools. Under a new law, anyone who supplied such tools would "receive a prison sentence and substantial fines".
"X or Grok could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for features—which responds to X users' prompts when they tag it in a message.
It is frequently used to provide commentary or more background to other users' remarks, but people on X are also able to edit an posted photo through its AI image editing function.
History of Controversy and Policy
The chatbot has been criticised for enabling users to create photos and videos with explicit content and sexualised material, and it was in the past accused of making sexually explicit content of public figures.
"The platform has been enabling the production and sharing of these images for a long time without taking any steps and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules bans "depicting likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was unlawful to "make or share private pictures or illegal content" and clarified this included explicit AI images generated with AI.
It said services such as X were obligated to take "suitable actions" to "reduce the danger" of users coming across unlawful material on their platforms and to delete it swiftly when they are notified of it.
