Optimizing Data Labeling for African Language AI: A Study on Annotation Budgets and Cross-Lingual Transfer

New research investigates the critical question for AI development in Africa: how much labeled data is truly needed for African languages, and can data from other African languages effectively substitute for it? The study provides empirical answers across 28 language-task pairs, covering news topic classification in 16 languages (MasakhaNEWS) and tweet sentiment analysis in 12 languages (AfriSenti).

The findings reveal that for news topic classification, approximately 400 labeled examples are sufficient to achieve 90% of the maximum performance in a median language. Sentiment analysis, however, proves more data-intensive, requiring thousands of labels for optimal performance across most languages. This highlights a significant difference in data demands depending on the specific AI task.

The research also explores the utility of pooling data from other languages. It demonstrates that cross-lingual pooling is highly beneficial when only a small budget of target language labels (e.g., 25) is available, significantly boosting performance. However, this advantage diminishes rapidly as the number of target labels increases, becoming negligible or even detrimental at larger datasets. This suggests a strategic approach to data utilization where pooling complements, rather than replaces, direct labeling efforts.

Crucially, the study provides concrete annotation guidance for teams developing African-language classifiers, particularly those operating without access to powerful GPUs. By releasing code that reproduces all results from public benchmarks, the authors empower developers to make informed decisions about data budgeting and cross-lingual strategies, ultimately accelerating the creation of effective AI solutions tailored for Africa's linguistic diversity.