OpenAI Debuts NCMEC-Built CSAM Classifier, ChatGPT Warning Screens
TL;DR
- OpenAI's new CSAM classifier flags 48% fewer images while identifying the same number of abusive ones, per TechCrunch.
- Images flagged by the classifier are twice as likely to be abusive, and the tool detects both authentic and AI-generated imagery.
- ChatGPT will suspend first-time offenders for 24 hours with a warning screen and permanently remove accounts for repeat violations.
OpenAI's new child sexual abuse material classifier flags 48% fewer images than existing tools while catching the same number of abusive ones, TechCrunch reports. That math means a flagged image is twice as likely to be genuine abuse, cutting the volume reviewers have to sit with.
The company said it developed the classifier with the National Center for Missing and Exploited Children and is working with the Tech Coalition and ROOST to share access through a free API with select companies. Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan of the University of Tasmania are also named as collaborators. The tool is built to catch both authentic and AI-generated imagery.
Inside ChatGPT, the policy change is more direct: "ChatGPT will show an alert to users attempting such prompts, direct users towards specialized support to stop harmful behaviour, and also suspend the account for 24 hours as a warning." Repeat violations trigger permanent account removal.
OpenAI is also publishing developer guidelines on preventing, detecting, and helping users report CSAM content, and is handing investigators dedicated plugins, investigative tools, and complex data analysis capabilities. The announcement lands the same day as separate findings that AI-linked child exploitation reports rose 8,411% in two years.
Originally reported by techcrunch.com
Read the original article →Original headline: OpenAI Launches CSAM Image Classifier With NCMEC and Ships ChatGPT Warning Screens and Account Bans