New 'Pearl-Chat' AI Model Engineered for Luganda Language, Outperforming Global Giants

Researchers have developed Pearl-Chat, an 8.5-million parameter language model specifically designed for Luganda, a low-resource Bantu language spoken by over 10 million people in East Africa. This initiative directly addresses the significant challenges faced by African languages in the AI landscape, where most large language models (LLMs) are built for high-resource Indo-European languages, leading to inefficiencies and poor performance for languages like Luganda.

The Pearl-Chat model was built from first principles using JAX and Flax NNX, incorporating advanced techniques such as explicit functional state decomposition and SPMD tensor parallelism. A key innovation is its custom byte-level byte-pair encoding (BPE) subword tokenizer, trained from scratch on curated Luganda corpora. This bespoke tokenizer achieves a remarkable 1.60 tokens per word fertility, a two-fold compression improvement compared to leading models like LLaMA-3 and Gemma.

The model underwent a two-stage training curriculum, involving pre-training on diverse Luganda texts and instruction fine-tuning on dialogues. It demonstrated strong performance on diagnostic benchmarks, achieving high ROUGE-L and BLEU scores while maintaining grammatical agreement across Bantu noun classes. The project underscores the potential for developing highly efficient and accurate AI solutions tailored to the unique linguistic structures of African languages.

All weights, tokenizers, and code for Pearl-Chat have been open-sourced, alongside a Python Server-Sent Events (SSE) streaming runtime. This open-source approach is crucial for fostering further research and development in African language AI, enabling local developers and researchers to build upon this foundation and create applications that are truly relevant and accessible to Luganda speakers.