A court in Amsterdam issued an injunction on Thursday prohibiting Elon Musk's chatbot Grok from generating AI-synthesized nude images of people and child sexual abuse material. xAI, the company that owns Grok, faces daily fines of approximately $115,000 if it fails to comply with the order. The company is already facing a lawsuit in Baltimore and is under investigation by the European Commission and the UK's cyber safety regulator, indicating escalating legal pressures.
A Dutch court has prohibited Elon Musk's xAI and its chatbot Grok from generating AI-synthesized images of adults and children without consent, intensifying the legal challenges confronting the firm. According to a machine translation of the court statement, the Amsterdam District Court issued a decree on Thursday banning xAI from producing and distributing "pornographic images depicting a person partially or completely nude without explicit permission." This marks the first ruling of its kind in Europe against Grok. The injunction also covers child sexual abuse material and prohibits the former Twitter, now X platform, from offering Grok's functionality within Europe. Under the order, xAI could be fined 100,000 euros (approximately $115,000) per day for non-compliance, with potential total penalties reaching up to 10 million euros. The case was brought by the Dutch non-profit organization Offlimits, which works to combat online sexual abuse, particularly targeting children and adolescents. The court also mandated that xAI pay 2.2 million euros in legal costs to Offlimits within 14 days. Robert Hoven, Executive Director of Offlimits, stated: "The judge drew a clear line yesterday: technology is not a free pass for online human rights violations. It is also encouraging that Europe voted yesterday to ban AI undressing tools. This is a significant double victory. People in every European country should enjoy the same rights online." CNBC has reached out to xAI for comment on the ruling but has not yet received a response. Data from the Center for Countering Digital Hate (CCDH) indicates that between December 29, 2026, and January 9, 2026, Grok generated approximately 3 million sexualized images, with over 23,000 of those images suspected to involve children. In January of this year, xAI implemented restrictions to prevent Grok from generating sexualized images of real people on the X platform, a measure applying to all users, including paying subscribers. However, Offlimits found these restrictions easy to circumvent, and the judge ruled that the protective measures were insufficient to adequately safeguard users. xAI is facing multiple lawsuits: On Tuesday, Baltimore became the first major US city to sue the company, accusing it of violating city consumer protection laws by falsely advertising Grok and the X platform as safe for users. The lawsuit references an online trend called "put a bikini on her," where users utilized Grok to create AI-undressed versions of images of people. xAI CEO Elon Musk himself participated in this trend, sharing an image generated by Grok depicting himself wearing a lace-up bikini. Additionally, three teenagers from Tennessee sued xAI last week after Grok generated content portraying them in sexual poses and scenarios. Concurrently, the European Commission launched an investigation into xAI in January, accusing it of disseminating non-consensual pornographic images of children and others. This case is being reviewed under the Digital Services Act (DSA). In the UK, the Office of Communications (Ofcom) and the Information Commissioner's Office have initiated separate investigations into the X platform and the issue of non-consensual pornographic imagery, respectively. While a pan-European verdict has not yet been delivered, countries in Southeast Asia, including Malaysia and Indonesia, blocked access to Grok in January due to the AI tool's persistent generation of non-consensual sexual content and "X's repeated failure to address the associated risks."
Comments