New AfriSwitch Benchmark Reveals Major Gaps in AI Speech Recognition for African Code-Switched Languages

Researchers have introduced AfriSwitch, a groundbreaking 61.36-hour dataset designed to benchmark automatic speech recognition (ASR) systems for code-switched African languages. Code-switching, the practice of alternating between two or more languages in a single conversation, is a common linguistic phenomenon across Africa, yet existing ASR technologies and evaluation benchmarks largely focus on monolingual speech. This new dataset addresses a critical gap by providing real-world, human-transcribed code-switched audio across 16 diverse African languages and varieties.

AfriSwitch includes detailed annotations such as English span tags for switched segments, a Code-Mixing Index (CMI) for each utterance, and counts of switch points, offering a nuanced understanding of code-switching behavior. The corpus statistics highlight significant variations in how African languages are mixed, indicating that no single metric can fully capture the complexity of code-switching within a language.

Initial evaluations using AfriSwitch on five leading open-source and commercial multilingual ASR systems revealed substantial performance deficiencies. Word Error Rates (WERs) were significantly higher than those reported for monolingual speech, with the best system achieving an average WER of 35.93%, and no system dropping below 24% on any language. This stark contrast underscores that current ASR models, despite their scale or stated language coverage, are ill-equipped to handle the linguistic diversity of African code-switching. The study suggests that targeted training on African code-switched data, rather than merely increasing model size or general multilingual coverage, is the key to improving performance.

This benchmark is crucial for advancing AI research and development in Africa, as it provides the necessary data to build more accurate and inclusive speech technologies. Improved ASR for code-switched African languages can unlock a wide range of applications, from voice assistants and transcription services to educational tools and accessibility solutions, directly benefiting millions of African speakers who naturally blend languages in their daily communication. It highlights the urgent need for localized AI development to meet the unique linguistic needs of the continent.