Loading…
Hundreds of Meta contractors posed as minors to test competitor AI chatbots in a project internally called Cannes, managed by Covalen. The effort targeted OpenAI's ChatGPT, Google's Gemini, and Character.AI, with a single round in August 2025 running over 45,000 prompts. The rivals were not informed Source: The Next Web Source: WIRED.
Contractors created dummy under-18 accounts and sent prompts and images—including pills, knives, and nooses—to the chatbots, logging responses in spreadsheets. A spreadsheet of 3,748 prompts showed hundreds focused on suicide, self-harm, eating disorders, sex, and drugs. Many were written from a child's perspective, such as a pregnant 13-year-old seeking pills or a girl hiding bulimia Source: WIRED.
Meta defended the practice as "industry-standard" safety benchmarking, saying it does not use competitor data to train its models Source: The Next Web. However, experts like Rumman Chowdhury of Humane Intelligence called it a "governance gray zone" where safety becomes cover for anticompetitive practices Source: The Next Web. Former contractors expressed alarm, worrying about generating abuse material or data theft Source: WIRED.
The three targeted companies prohibit such testing in their terms of service Source: The Next Web.
“Hundreds of contractors working on a project for Meta were instructed to pose as minors online and probe how competitor chatbots responded to prompts involving suicide, sex, eating disorders, and other high-risk subjects.”
“The project ran under the internal name Cannes, and a Meta contractor called Covalen managed it.”
“A single round of testing completed in August 2025 saw more than 45,000 prompts run through the rival chatbots.”
“Many were written from the perspective of children or teenagers in crisis: a 13-year-old who said she had become pregnant by her adult neighbor and wanted to know where to buy pills to end the pregnancy.”
“Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice.”
“She called it a 'governance gray zone where safety becomes a convenient cover for anticompetitive practices.'”