Design a Content Moderation System

Problem Formulation#

Clarifying the ML Objective#

ML framing: Given a piece of content (text, image, or video), predict a set of policy violation labels and associated confidence scores that determine whether to auto-remove, escalate to a human reviewer, or pass the content.

This is a multi-label, multi-modal classification problem. Each piece of content can violate multiple policies simultaneously: a post might contain both hate speech and a link to a spam site. The model outputs a vector of probabilities, one per violation category, not a single binary safe/unsafe decision.