🔗 Share this article

The social media platform X has provoked major backlash after people used its AI chatbot Grok to manipulate photos of females by stripping their garments.

Numerous instances have been seen where the technology has been used to digitally undress women to render them in bikinis without consent, as well as to put them in sexually explicit situations.

Developer Reaction and Regulatory Position

XAI, the organization responsible for Grok, offered no comment to inquiries, other than an auto-reply response stating "legacy media lies".

A government official stated that legislation is being proposed to outlaw nudification technology. Under a new law, anyone who provided such technology would "face a jail term and significant penalties".

"X or Grok could prevent these types of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."

Regulator Stance and Platform Functionality

The regulator Ofcom said technology companies must "evaluate the danger" of people accessing illegal content on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated pictures.

Grok is a no-cost AI assistant—with some additional functions—which answers X users' queries when they reference it in a tweet.

It is often used to offer reactions or more context to other posters' remarks, but people on X are also can alter an posted photo through its AI image editing function.

Background of Criticism and Policy

The chatbot has been faulted for enabling users to create photos and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic content of celebrities.

"The platform has been allowing the creation and distribution of these pictures for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.

XAI's own usage rules prohibits "portraying likenesses of people in a pornographic way".

In a comment, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes generated with AI.

It said platforms such as X were obligated to take "suitable measures" to "lessen the danger" of users encountering unlawful material on their services and to remove it swiftly when they learn of it.