Incident sheet · /security

Filed
Read
3 min · 650 words
Severity
Routine

OpenAI releases image classifier for AI-generated CSAM

OpenAI's free CSAM image classifier, built with NCMEC, reached 99.1% offline precision but 29.8% on ChatGPT traffic, raising questions about live review workloads.

OpenAI releases new image classifier to detect CSAM, including images generated by AI
FeedOpenAI releases new image classifier to detect CSAM, including images generated by AI — 14398 / pixabay

Event log

  1. The model card, published November 16, 2026, covers an image classifier OpenAI developed with NCMEC and offers free to approved organisations.
  2. The architecture combines an internal CLIP image encoder with a separate three-layer machine-learning model.
  3. Offline evaluation recorded 99.1% precision and 80.2% recall for the likely-CSAM score.
  4. The AI-generated CSAM score reached 100% precision and 86.4% recall offline.
  5. ChatGPT-traffic precision fell to 29.8% for CSAM and 34% for AI-generated CSAM.

By Daniel Okafor

OpenAI’s suspected child sexual abuse material (CSAM) score hit 99.1% precision offline but only 29.8% on ChatGPT traffic, according to the model card published November 16, 2026. OpenAI released the tool free to approved organisations. The company developed it with the US-based National Center for Missing and Exploited Children.

What does the classifier actually detect?

The classifier assigns risk scores to suspected CSAM so trained reviewers can prioritise images for investigation. A score alone does not establish whether an image is CSAM, its legal status, or any organisation’s reporting obligation. OpenAI cautions against treating the output as a legal determination.

Hash matching compares images with a database of previously identified CSAM. It can miss newly created, altered or AI-generated material without a matching hash. OpenAI’s classifier analyses visual content instead, seeking novel or manipulated imagery that hash matching may miss.

How does the scoring work?

An internal CLIP image encoder converts each image into a visual representation. A separate three-layer machine-learning model then converts that representation into risk scores.

OpenAI said it selected this architecture for its ability to “generalise to previously unseen material” while operating quickly.

The model produces two scores: one for suspected CSAM and another for suspected AI-generated CSAM. Those scores can both apply to the same image. Neither score proves AI authorship, and the AI-generated score does not verify an image’s provenance.

Who can use the classifier?

Approved access covers three organisation types:

  • Platform trust and safety teams.
  • Non-governmental organisations working to prevent child sexual abuse and exploitation.
  • Law enforcement agencies.

Authorised organisations can apply the tool across five operational tasks:

  • Identifying potentially novel, manipulated or AI-generated CSAM.
  • Prioritising images for trained human review.
  • Supporting content moderation and account-enforcement decisions.
  • Assisting investigations and reporting workflows.
  • Monitoring patterns of abuse.

What restrictions apply before deployment?

OpenAI says organisations should “not submit images already confirmed as CSAM” to the classifier. They should instead follow their established response and reporting procedures.

The model assists human reviewers and does not replace them. Organisations considering preventive blocking without human review must separately assess applicable legal, contractual and operational requirements.

How reliable are the scores?

OpenAI’s model card reports four headline evaluation results:

  • The likely-CSAM score achieved 99.1% precision and 80.2% recall offline. Roughly 99 of every 100 CSAM flags were correct, while four in five actual CSAM examples were caught.
  • The AI-generated CSAM score achieved 100% precision and 86.4% recall offline. Every flagged AI-generated CSAM image in that test was correct, and about 86 of every 100 such examples were caught.
  • ChatGPT-traffic precision for the CSAM score fell to 29.8%. Roughly 30 of every 100 CSAM flags were correct.
  • ChatGPT-traffic precision for the AI-generated CSAM score was 34%. Roughly one in three flags proved correct.

Precision measures how often a flag proves correct. Recall measures how much actual material the system finds.

OpenAI attributes part of the online gap to the low prevalence of CSAM in real-world traffic, where false positives can make up a larger proportion of flagged images. The company says the online results are more indicative of the workload reviewers may face.

What remains unproven?

The classifier can produce false positives and false negatives. Its performance may also vary with the content and images that different platforms encounter.

The company advises every organisation to test it against its own data before deployment. The card gives no test-set size, deployment count, throughput figure or model version.

OpenAI has not evaluated the model on video frames. Human review remains necessary for reporting decisions, despite the 100% offline precision for the AI-generated score.

Deployment therefore still rests on trained reviewers, local testing and established reporting processes. The tool sets review priority; it does not settle legal or policy questions.

via medianama.com (Source)

Filed under

  • openai
  • csam-detection
  • ai-generated-content
  • content-moderation
  • child-safety
Share this article:

More from Daniel Okafor

Daniel Okafor

Show full bio

Staff writer covering technology and product news in enterprise information technology and the leaders who run at CIO Monitor.

203 articles

The newsroom digest

One email a day with the stories that matter. No noise.

By subscribing you agree to the privacy policy.

Cross-references

‹ Previous articleNext article ›