xAI sues Grok user over child sexual abuse material deepfakes

xAI has filed a lawsuit against Terry Wayne Harwood, a South Carolina man, for allegedly using its Grok chatbot to generate child sexual abuse material (CSAM) and non-consensual sexual deepfakes of adults and minors. The complaint, filed in federal court in Texas on Tuesday, marks one of the first lawsuits by an AI company against a user for misuse of its system. Source: CNN

The Allegations

According to xAI, Harwood opened multiple accounts using false identities and submitted "misleading prompts" to circumvent Grok's built-in safeguards. The company says Grok refused many attempts, but Harwood repeatedly revised his prompts to bypass moderation. Examples include uploading a photo of a fully clothed 10-11-year-old girl and asking Grok to remove her clothing and depict her in a pornographic manner. Source: Engadget Harwood was arrested in February on state charges of sexual exploitation of a minor. Source: The Verge

xAI's Enforcement Figures

The lawsuit reveals that xAI has suspended over 52,000 accounts and filed more than 73,600 reports to the National Center for Missing & Exploited Children (NCMEC) in 2026, leading to at least 244 arrests. Source: Al Jazeera xAI argues this demonstrates its commitment to enforcing rules, while critics note that Grok previously allowed "digital undressing" through a "spicy mode" and that Musk himself said in January he was unaware of any underage images generated by Grok. Source: Gizmodo, The Next Web

Legal and Regulatory Context

xAI faces parallel lawsuits and investigations in the UK, EU, and US over Grok's output. The company is seeking unspecified damages and a permanent ban preventing Harwood from using the platform. The case highlights the tension between AI companies' responsibilities and user accountability.