Explaining Twitter’s inability to effectively moderate content during the COVID-19 pandemic

Social media platforms routinely face pressure to restrict harmful content while protecting free speech; however, prior theory suggests that platform design might undermine the efficacy of content moderation. During the COVID-19 pandemic, major social media platforms removed content that violated their medical misinformation policies. Although controversial, it is widely assumed that these interventions, such as deplatforming and content removal, are efficacious; however, this claim has not been evaluated based on evidence. We therefore evaluated the efficacy of Twitter’s attempts to curtail vaccine misinformation during the COVID-19 pandemic. We found that several clusters of vaccine skeptical accounts generated a larger share of tweets about vaccines, increased in virality, and continued to spread low-quality and likely misinformative content despite Twitter’s sustained removal of content and accounts — both when compared to prior trends and to corresponding clusters of pro-vaccine accounts. Several of these accounts were subject to a contemporaneous moderation action, in which Twitter removed 70,000 accounts on January 8, 2021. Although this action reduced activity among targeted accounts, virality increased and information quality decreased among both these accounts and those sharing similar political affiliations, calling into question the efficacy of these removals. Novel platforms that share Twitter’s architecture may therefore face similar moderation challenges.