Misinformation detection on YouTube using video captions

Millions of individuals use online social media platforms, such as YouTube, Facebook, and X (formerly Twitter), to share information. However, these platforms have also been used to establish a narrative, to conduct propaganda, and to disseminate misinformation. Among all, YouTube is particularly notable, as it hosts the world’s largest collection of videos uploaded by individuals and has been reported to be one of the biggest sources of misinformation. In this work, we propose a framework that utilizes video captions (subtitles) to classify videos into three classes: Misinformation, Debunking Misinformation, and Neutral. The motivation behind exploring video captions stems from our analysis of video metadata, which revealed that attributes such as view counts, likes, dislikes, and comments are ineffective, as videos are difficult to differentiate using this information. The framework achieved an F1 score of 0.90 and 0.96 using deep sequential and state-of-the-art large language models (LLMs), respectively. The framework also uses an explainable Attention-based approach to analyze critical sections of the video that trigger classification. Additionally, to address the problem of insufficient misinformation datasets, we created and host a fully functional crowd-sourcing platform for the dataset collection, where users can submit annotated YouTube videos. We demonstrate the practical utility of this platform by contributing to the misinformation dataset that has been collected using this platform.