AI at the Front Lines of Platform Governance: Using LLMs to Support Illegal Content Reporting under the Digital Services Act

Illegal content reporting mechanisms are a key technical and organizational measure through which online platforms address the dissemination of illegal content under European Union law. Under the Digital Services Act (DSA), user notices submitted pursuant to Article 16 must be sufficiently substantiated and provided in good faith, requiring users to interpret legal and procedural language and translate it into legally meaningful categories and reasons. In practice, however, reporting illegal content remains cumbersome across major social media platforms, placing substantial cognitive and legal demands on users. Without effective support at the reporting interface, operationalizing Article 16 in practice remains challenging. We investigate how large language model (LLM)-based assistants can support illegal content reporting. In a controlled user study (N = 450) using an interface modeled on a major platform’s reporting workflow, we compare three conditions: (1) a conventional explainable AI assistant (XAI) that suggests a single legal category with a rationale, (2) an evaluative AI assistant (EvalAI) that presents balanced pro and con arguments across candidate legal provisions for user deliberation, and (3) a baseline reflecting unaided reporting (Baseline). We further examine these assistance forms under systematically varied AI error regimes. Our results show that EvalAI improves provision-level accuracy under AI error regimes and reduces misclassification distance relative to conventional XAI, particularly for near-miss and overbreadth errors. In contrast, conventional XAI does not improve—and can degrade—the quality of users’ rationales relative to unaided reporting, despite enabling faster decisions when the AI output is correct. We discuss implications for the design of compliance-oriented reporting interfaces, highlighting trade-offs between accuracy, deliberation, and vulnerability to misleading AI output.