MANILA, Philippines – OpenAI announced the release of a new image classifier that can detect potential child sexual abuse material (CSAM), including AI-generated material.
The classifier can identify and help detect CSAM images, including those generated by AI. OpenAI said their new classifier “looks for visual signs of abuse, helping teams identify novel images that may not have been seen before.”
The classifier assigns scores to images it spots, helping content reviewers decide which images may require attention or intervention, and OpenAI said this will be made available to select companies through its API with no cost.
The company added that theirs is “the first CSAM classifier with broad availability that has a specific signal on indicating if CSAM may be AI generated.”
In early testing with partners, OpenAI said the new image classifier flags 48% fewer images but finds the same number of abusive images. This means a potential reduction of work for reviewers, as it noted that images flagged by the classifier are apparently twice as likely to contain abuse.
Intervention
OpenAI also said it is implementing “product warnings in ChatGPT that interrupt attempts to find abusive material, explain the consequences, and point people toward help to stop harmful behavior.”
The company said their warning makes it clear that “sexualizing children causes harm, including when the requested material…

