New Research Improves AI Text Classification for Low-Resource African Languages

A new research paper highlights a critical flaw in current AI text classification systems, particularly for the diverse linguistic landscape of the Global South, including Africa. These systems often use a single confidence threshold across multiple languages, leading to inconsistent accuracy. While a pooled threshold might meet an average target, it significantly underperforms for specific low-resource languages, such as Somali and Tigrinya, as demonstrated on the MasakhaNEWS and AfriSenti datasets which feature numerous African languages.

The study proposes a more effective approach: estimating a unique confidence threshold for each language. This method ensures that every language achieves the desired accuracy target (e.g., 90%) without requiring model retraining. The research underscores the varying costs of this promise, revealing that some languages like Amharic and Xitsonga require a much higher rate of human review (over 80% of tweets) compared to others like Nigerian Pidgin (under 8% of news).

This finding is particularly significant for African contexts, where AI deployments often operate on limited computational resources and serve a multitude of languages with scarce labeled data. The suggested fix—calibrating, reporting, and budgeting human review language by language—is presented as affordable and efficient, requiring only one to two hundred labels per language and minimal training time on standard CPUs. This ensures more reliable and equitable AI performance across Africa's rich linguistic diversity.