AI System Developed to Automate South Africa's Food Label Warning Screening

Researchers have developed an AI-powered workflow to automate the screening of food packages against South Africa's draft R3337 regulations, which mandate front-of-package warning labels for unhealthy foods. This system aims to streamline the process of identifying foods high in sugar, sodium, and saturated fat, a critical step for public health initiatives.

The workflow utilizes a three-class detector to locate key information on food packaging, such as nutrition panels and ingredient lists. Optical Character Recognition (OCR) then extracts the relevant data, which is fed into a rule-based system to assess compliance with R3337. A significant feature is its abstention mechanism, which flags cases of incomplete or conflicting evidence for human review, rather than forcing an uncertain automatic decision.

For complex or ambiguous cases, the system incorporates a selective vision-language model (VLM) review stage, using Qwen3-VL 8B Instruct, to re-evaluate unresolved evidence. This hybrid approach, combining deterministic rules with VLM capabilities, aims to improve both the coverage and accuracy of the screening process, ensuring reliable identification of non-compliant products.

Applied to a dataset of 442 products, the system demonstrated high accuracy, particularly after the VLM review stage resolved additional cases. While the VLM improved coverage, the study highlighted the continued importance of deterministic rules and explicit abstention for maintaining reliability, as the VLM still produced some incorrect evidence extractions. This research is crucial for the efficient implementation of South Africa's food labeling policy, potentially setting a precedent for similar public health initiatives across the continent.