Amplifying Global Voices: Building a Localized Data Platform for Globalized AI

Current AI models often fail to account for local context and language, given the predominance of English and Western internet content in their training data. This hinders the global relevance, usefulness, and safety of these models as they gain more users around the globe. Here we present a data platform and methodology that utilizes Subject Matter Experts from different communities to build a diverse dataset. The data produced through this method is intended to probe models for limitations, thus challenging models in ways that are both culturally specific and on distinct subject areas. Through a pilot conducted in Sub-Saharan Africa (Ghana, Kenya, Malawi, Nigeria, South Africa, and Uganda), this approach resulted in a dataset of 10,044 queries across seven languages (e.g., Luganda, Swahili, Chichewa) that was co-created with 213 Subject Matter Experts (e.g., health workers, teachers). By developing a safety classifier using this data, we show how targeted, expert-generated datasets can increase the average accuracy of the base Gemma model from 32% to 75% in detecting culturally unsafe queries related to Sub-Saharan Africa. We release the dataset and code as part of this paper. CONTENT WARNING: This paper contains examples of prompts that may be offensive.