Vambo AI Trains African-Language Model Amid Data Shortage

Vambo AI has built a 1.5-billion-parameter artificial intelligence model covering 12 African languages, English, French and code, while confronting a persistent shortage of reliable African-language training data.

The Johannesburg-based company trained MORENA from scratch using access to CINECA’s Leonardo supercomputer through the United Nations Development Programme’s AI Hub for Sustainable Development. Some training runs used 512 GPUs simultaneously.

According to TechCabal’s report, the computing access meant Vambo’s cash cost for the training was zero. The project used about 22,000 A100 GPU-hours, estimated in the report to have a value of between $25,000 and $40,000.

Limited supply of local-language data

Isheanesu Misi, Vambo AI’s co-founder and chief technology officer, said the company struggled to find enough real text in African languages to train the model.

“Finding that much real data was close to impossible, so a lot of it was synthetic,” Misi said.

MORENA covers ChiShona, Kiswahili, Hausa, Yorùbá, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and Nigerian Pidgin. Its training data contained 251.7 billion tokens, including 65.6 billion tokens in African languages.

Misi said synthetic data helped Vambo guide the model towards capabilities such as tool calling, but it also introduced limitations. “You would find that some of the grammar of the outputs is close but imperfect,” he said.

He also pointed to the limited resources available for several African languages. Languages including isiNdebele and Nigerian Pidgin, he said, have “next to no resources available” for AI development.

Reported performance and access

Vambo’s model documentation reported that MORENA recorded a score of 1.408 bits per byte across the 12 African languages, the lowest score among 26 models tested in the comparison. The company presented this as evidence of strong performance in African-language text modelling, although the results were not independently verified in the report.

Vambo said MORENA’s tokenizer represented African-language text using 1.39 times fewer tokens than Gemma 3 and 1.53 times fewer tokens than Llama 3.2.

The model weights have been released under an Apache 2.0 licence, allowing developers to use them as a base for further work. The training corpus had not been released at the time of the report.

Smaller versions with 0.5 billion and 0.2 billion parameters were designed for lower-resource environments. Vambo also positioned MORENA as a foundation that developers could fine-tune for applications such as fintech and education rather than build a new large model from the beginning.

Infrastructure remains a challenge

Misi said access to Leonardo’s computing capacity was essential to the project. “Without access to that kind of compute power, this project would have been dead on arrival,” he said.

At the same time, he questioned whether similar projects could be repeated without subsidised access to advanced computing infrastructure. “If everyone is to try and build a model from scratch, it’s prohibitively expensive, complicated, time-consuming,” Misi said.

He argued that fine-tuning existing models could offer a more practical route for developers, estimating that it could sometimes cost $100 or less.

The project illustrates two linked barriers facing African-language AI development: access to powerful computing systems and the lack of high-quality, documented and scalable language data. Africa has more than 2,000 languages, while many have limited digitised text and speech resources.

Other efforts are also addressing the gap. Google’s WAXAL project is described as an open-source speech dataset covering 21 sub-Saharan African languages, while initiatives involving UduTech, Chipmango and BARO AI are working on computing and semiconductor capacity in Africa.