Development and validation of a tool for detecting misinformation risk in diet, nutrition, and health content (Diet-MisRAT)

Misinformation in diet and nutrition is recognised as a major public health threat, with the potential to misguide dietary choices and contribute to preventable harm. To address this, we developed a Misinformation Risk Assessment Model (MisRAM), grounded in the World Health Organization’s hazard risk assessment principles. MisRAM conceptualises misleading content traits as stratifiable agents of informational adverse effects, weighed by their severity and likelihood of increasing recipient susceptibility. Building on this model, we designed the Diet-Nutrition Misinformation Risk Assessment Tool (Diet-MisRAT), a structured instrument that evaluates medium-to-long form content across four risk dimensions (inaccuracy, incompleteness, deceptiveness, health harm), yielding five-tier risk estimates from very low to very high. Validation involved five rounds: expert reviewers, trainee dietitians, postgraduate nutrition students, highly experienced nutrition professionals, and zero-shot prompt-based generative-AI risk detection. Results showed strong to very strong alignment with expert-derived benchmarks, supporting the tool’s interpretability and concurrent validity. ChatGPT demonstrated high test–retest reliability, accuracy, precision, sensitivity, and F1 scores under blinded untuned conditions, suggesting that adequately constructed, expert-designed prompting tools may help overcome training-dataset limitations. Diet-MisRAT offers a scalable, graded alternative to binary detection. Domain-calibrated risk stratification could guide proportionate interventions in content oversight, regulation, education, misinformation inoculation, and infodemic mitigation.