🔗 Share this article

The social media platform X has provoked widespread outrage after individuals utilized its artificial intelligence assistant Grok to alter images of females by stripping their attire.

Several cases have emerged where the system has been employed to virtually strip female subjects to render them in swimwear without consent, as well as to put them in sexually explicit scenarios.

Company Response and Legal Stance

XAI, the organization behind Grok, remained silent to requests for comment, except for an auto-reply message stating "traditional news outlets lies".

A government official stated that legislation is being brought forward to prohibit nudification software. Under a proposed law, anyone who provided such technology would "be subject to a prison sentence and significant penalties".

"X or Grok is capable of halting these forms of misuse if they chose to," said a law professor, adding they "operate with impunity."

Oversight Body Position and Tool Functionality

The regulator Ofcom said tech firms must "evaluate the potential" of people viewing illegal content on their platforms, but did not confirm whether it was presently examining X or Grok in regarding AI-generated images.

Grok is a free chatbot—with some paid-for features—which responds to X users' requests when they tag it in a message.

It is commonly used to provide reactions or further background to other users' comments, but people on X are also able to edit an posted image through its artificial intelligence picture modification tool.

History of Controversy and Policy

The chatbot has been condemned for permitting users to create images and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in generating inappropriate material of public figures.

"X has been enabling the production and spread of these images for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.

XAI's own terms of service prohibits "depicting likenesses of people in a pornographic way".

In a comment, Ofcom said it was against the law to "make or circulate deepfakes or illegal content" and clarified this covered sexual deepfakes made with AI.

It said networks such as X were obligated to take "suitable steps" to "minimize the risk" of users viewing illegal material on their platforms and to take down it promptly when they are notified of it.