🔗 Share this article

The online network X has caused significant outrage after users employed its AI chatbot Grok to alter photos of women by removing their attire.

Several cases have emerged where the system has been used to remove clothing from women to show them in bikinis without permission, as well as to place them in sexually explicit contexts.

Developer Reaction and Legal Position

XAI, the company that created Grok, offered no comment to questions, other than an standard response stating "traditional news outlets lies".

A government spokesperson stated that new laws is being proposed to ban image stripping software. Under a new law, anyone who supplied such tools would "receive a prison sentence and significant fines".

"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."

Regulator Position and Tool Capabilities

The regulator Ofcom said technology companies must "evaluate the danger" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated images.

Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' requests when they mention it in a tweet.

It is frequently used to provide reactions or additional background to other posters' comments, but people on X are also capable of alter an uploaded picture through its AI image editing tool.

Background of Criticism and Rules

The system has been criticised for enabling users to produce images and footage with nudity and sexualised material, and it was earlier accused of making graphic content of celebrities.

"X has been permitting the creation and distribution of these images for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.

XAI's own acceptable use policy forbids "showing likenesses of individuals in a graphic fashion".

In a comment, Ofcom said it was illegal to "generate or distribute deepfakes or illegal content" and confirmed this included sexual deepfakes created with AI.

It said platforms such as X were required to take "suitable actions" to "minimize the risk" of users coming across unlawful content on their platforms and to delete it quickly when they are notified of it.