African Languages Expose Flaws in LLM Safety Transfer

New research reveals that safety mechanisms in large language models (LLMs), primarily developed in English, fail to adequately protect users in low-resource African languages. The study challenges the common assumption that safety alignments generalize across different languages, highlighting a significant vulnerability for non-English speakers.

Researchers utilized a novel dataset called LoDNA, which includes both literal translations and culturally localized prompts, to test safety transfer in Twi, Hausa, Amharic, and Swahili. Instead of relying solely on generated text, they developed a latent geometric framework to analyze how LLMs process refusal signals at a deeper, hidden-state level, providing a more robust evaluation of cross-lingual safety.

The findings are stark: harmful prompts in these African languages retained less than 10% of the English refusal signal across most model configurations. While literal and localized prompts showed high semantic alignment, the safety mechanisms did not activate effectively. This suggests that even when models understand the content, they fail to route it through their safety protocols, indicating a superficial multilingual alignment.

This research is crucial for Africa as it underscores the urgent need for localized safety development in AI. As LLMs become more integrated into daily life across the continent, the lack of robust safety in local languages could lead to the proliferation of harmful content, misinformation, and biased outputs, disproportionately affecting African populations. It calls for a fundamental shift from English-centric safety paradigms to inclusive, language-specific approaches.

The study provides strong evidence against the notion of a universal, language-agnostic harm detection system, emphasizing that current multilingual safety measures are insufficient. This has profound implications for AI developers and policymakers working on AI deployment and regulation in African contexts, demanding dedicated efforts to ensure equitable and safe AI experiences for all language communities.