India asks X to address obscene content generated by Grok

The Indian government has ordered Elon Musk-owned blogging platform X to make changes to its Grok artificial intelligence chatbot over obscene AI content. The country wants the platform to make immediate technical and procedural changes to the chatbot after users and lawmakers flagged several instances of obscene content.

According to the country, the cases of the chatbot being used to create these obscene images, including AI-altered images of women created using the tool. The order was issued on Friday by the IT ministry of India, asking the platform to take corrective actions concerning Grok, including restricting the generation of content involving “nudity, sexualization, sexually explicit, or otherwise unlawful” material. The ministry has also given the platform a 72-hour ultimatum to effect the changes.

Indian government orders X to make technical changes to Grok

As part of its deliverables in the 72-hour ultimatum, the ministry has also asked X to submit an action-taken report detailing the steps it has taken to prevent the hosting or dissemination of indecent content. These contents include “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise prohibited under law.” The order warned that if the platform fails to carry out the mandatory steps, it could jeopardize its “safe harbor” protections.

India’s order comes amid concerns raised by users who shared examples of Grok being prompted to alter images of individuals, primarily women, to make them appear to be wearing bikinis. The issue was prompted by a formal complaint, which was submitted by Indian parliamentarian Priyanka Chaturvedi. Chaturvedi cautioned all big firms with AI chatbots, adding that there have to be guardrails put in place to check features like Grok so the chatbot does not violate human dignity.

Source: @priyankac19 via X.

Separately, users have also flagged several incidents where the AI chatbot has been used to generate sexualized images involving minors, an issue which X acknowledged on Friday. The platform claimed that it was caused by some lapses in safeguards before moving swiftly to take down the generated images. The platform also urged users to report formally to the Federal Bureau of Investigation (FBI) or NCMEC’s CyberTipline, noting that X is committed to preventing such issues.

X could face legal issues for non-compliance

According to users, indecent images of women generated using the chatbot remained on the platform. The latest order comes after the Indian IT ministry issued an advisory to social media platforms, reminding them that compliance with local laws governing obscene and sexually explicit content is a prerequisite for retaining legal immunity from liability for user-generated materials. The country urged companies to improve guardrails and warned that failure could invite legal action.

“It is reiterated that non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers, and the users on the platform who violate the law, without any further notice,” the order warned. The Indian government has also warned X that if it fails to comply with its directive, the country has the power to initiate a legal action against the platform under India’s IT law and criminal statutes.

India has emerged as one of the biggest digital markets, with the country being seen as a critical test case for how far governments are willing to push in order to hold platforms responsible for AI-generated content. If enforcement is tightened in the country, it could have a ripple effect for global technology companies operating across other jurisdictions. The order also comes as X continues to challenge India’s content regulation rules in court.

The platform argues that federal government takedown powers risk an overreach, even as the platform has complied with most of the blocking directives put forward by the country. At the same time, Grok has been helpful to users on X, with most users using the platform for real-time fact-checking and commentary on global events, making its output more visible and more politically sensitive than that of stand-alone AI tools.

Don’t just read crypto news. Understand it. Subscribe to our newsletter. It's free.