Elon Musk's Grok faces global scrutiny for sexualised AI deepfakes
xAI said on 14 January it had restricted image editing for Grok AI users and blocked users, based on their location, from generating images of people in revealing clothing in 'jurisdictions where it's illegal'
Governments and regulators around the world are cracking down on sexually explicit content generated by Elon Musk's xAI chatbot Grok on X, launching probes, imposing bans and demanding safeguards in a growing global push to curb illegal material.
Here are some reactions from governments and regulators in January:
Europe
Keep updated, follow The Business Standard's Google news channel
The European Commission on 26 January opened an investigation into whether Grok disseminates illegal content such as manipulated sexualised images in the EU. The probe will examine whether X properly assessed and mitigated risks as required under the bloc's digital rules.
The Commission had on 8 January extended a retention order sent to X last year to retain and preserve all internal documents and data related to Grok until the end of 2026.
Britain's media regulator Ofcom has launched an investigation into X to determine whether sexually intimate deepfakes produced by Grok violated its duty to protect people in the UK from content that could be illegal, under the country's Online Safety Act framework.
In France, government ministers referred sexually explicit Grok-generated content circulating on X to prosecutors and alerted French media regulator Arcom to check the platform's compliance with EU rules.
Germany's media minister Wolfram Weimer also said EU rules provided tools to tackle illegal content and alleged the problem risked turning into the "industrialisation of sexual harassment".
Italy's data protection authority warned that using AI tools to create "undressed" deepfake imagery of real people without consent could amount to serious privacy violations and, in some cases, criminal offences.
Swedish political leaders have also condemned Grok-generated sexualised content after reporting that imagery involving Sweden's deputy prime minister was produced from a user prompt.
Asia
India's IT ministry sent X a formal notice on 2 January over alleged Grok-enabled creation or sharing of obscene sexualised images, directing the content to be taken down and requiring a report on the actions being taken within 72 hours.
Japan also probed X over Grok, stating that the government would consider every possible option to prevent the generation of inappropriate images.
Indonesia's communications and digital ministry said it had blocked access to Grok, a move digital minister Meutya Hafid said was meant to protect women and children from AI-generated fake pornographic content, citing Indonesia's strict anti‑pornography laws.
Malaysia restored access to Grok for its users after X implemented additional safety measures, its communications regulator said on 23 January.
The Philippines will reinstate access to Grok after its developer pledged to remove image-manipulation tools that had sparked child-safety concerns, the country's cybercrime investigation unit said on 21 January.
Americas
California's governor and attorney general said on 14 January they were demanding answers from xAI amid the spread of non-consensual sexual images on the platform.
Canada's privacy watchdog said it was widening an existing investigation into X after reports that Grok was generating non-consensual, sexually explicit deepfakes.
Brazil's government and federal prosecutors gave xAI 30 days to prevent the chatbot from spreading fake sexualised content, according to a joint statement on 20 January.
Oceania
Australia's online-safety regulator eSafety said on 7 January it was investigating Grok-generated "digitally undressed" sexualised deepfake images, assessing adult material under its image‑based abuse scheme and noting current child-related examples it had reviewed did not meet the legal threshold for child sexual abuse material under Australian law.
How has xAI responded?
xAI said on 14 January it had restricted image editing for Grok AI users and blocked users, based on their location, from generating images of people in revealing clothing in "jurisdictions where it's illegal". It did not identify the countries.
It had earlier limited the use of Grok's image generation and editing features only to paying subscribers.
