OpenAI Unveils New Classifier and ChatGPT Restrictions to Combat Child Abuse Material

OpenAI has announced a suite of new tools and safety measures designed to detect and prevent the creation of child sexual abuse material (CSAM), including content generated by artificial intelligence. The company is introducing a new image classifier that identifies visual signs of abuse in previously unseen images, assigning scores to help human reviewers prioritize content. Developed in collaboration with the National Center for Missing and Exploited Children (NCMEC), the classifier is being offered to select companies through its API at no cost. Early testing indicated the tool flags 48% fewer images for review while maintaining the same detection rate for abusive material, and flagged images were twice as likely to be abusive.

In addition to the classifier, OpenAI is implementing new deterrents within ChatGPT to prevent users from accessing or generating sexualized content involving children. Users attempting to access such material will encounter warning screens that explain the harm caused by such behavior and direct them toward specialized support. These warnings are accompanied by a 24-hour account suspension, and repeated offenses may result in permanent account removal. The company stated that these guidelines were developed alongside researchers and organizations including the Lucy Faithfull Foundation, Protect Children, and Joel Scanlan of the University of Tasmania.

To support broader industry efforts, OpenAI is publishing guidelines for developers to help them detect and report CSAM. The company is also providing law enforcement agencies with AI-powered investigative tools, dedicated plugins, and complex data analysis capabilities. These resources include a field guide and training for investigators. The initiative follows increased scrutiny of AI technology, including reports from NCMEC that its CyberTipline received more than 400,000 reports involving generative AI in 2025.

Other organizations have participated in evaluating the new technology. The AI platform Fal reported that its reviewers confirmed 72% of high-confidence flags as previously unidentified CSAM, while SafetyKit reported the classifier detected 90% of cases involving sexualized drawings or animations in its evaluation.

FI Journal
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.