Egyptian English and Nigerian English have joined the Open Automatic Speech Recognition leaderboard.
The Open Speech Recognition Leaderboard now includes private Egyptian and Nigerian English speech data from Liva AI. The update expands earlier private splits to reduce benchmark gaming and better reflect real users. Results are grouped using Kachru's Three Circles, separating US and non-US accents and Inner Circle accents from Outer and Expanding Circle accents. The goal is more trustworthy speech recognition evaluation across global English varieties.
Key Takeaways
- The Open ASR Leaderboard added private Egyptian and Nigerian English test data with Liva AI to improve accent diversity.
- Held-out private data reduces benchmark overfitting and makes model comparisons more trustworthy.
- Results will be reported using Kachru’s Three Circles, comparing US, Inner Circle, and Outer/Expanding Circle accents.
- Liva AI contributes natural speech data from real consumer interactions rather than only task-based recordings.
- Future rankings may change as models must prove performance across more accents, languages, and private datasets.
Meta description: Open ASR Leaderboard adds private Egyptian and Nigerian accents, plus Kachru-based breakdowns, to make automatic speech recognition testing more trustworthy.
Tags: Automatic Speech Recognition evaluation, speech recognition, AI benchmarks, private test data, accent diversity
Featured image: abstract, friendly illustration of speech waveforms joining global accent groups, no people, logos, or text
What the update delivers
The Open ASR Leaderboard has announced a new collaboration with Liva AI to add private evaluation data for two new accents: Egyptian and Nigerian. For researchers, speech technology teams, and developers building voice products, this means the leaderboard will include more held-out speech samples that test whether automatic speech recognition systems can handle English across a broader range of accents.
The update extends a private-data push that began a few months ago. At that time, the leaderboard added 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech from five accents, as described in an earlier blog post. The response from the evaluation community was strong, with many participants highlighting why held-out sets are needed for trustworthy model comparisons. After that change, the project added one more private split for Indian-accented English with Voice Arena, also announced in a blog post.
Why held-out data changes trust
Public benchmark sets can become less useful when teams know exactly what is being tested. Systems may be tuned, selected, or compared in ways that improve leaderboard scores without proving that the model can understand speech from different real users. The Open ASR Leaderboard describes the new private data as a way to minimize benchmaxxing, while also improving representation by adding more accents and more languages.
This matters for anyone using benchmark results to choose a model, because a system that performs well on visible data may not perform as well on speech that users actually produce. Private splits give evaluation teams a cleaner signal: if a model remains accurate when the test material is not in the public set, the result is more likely to reflect general speech understanding rather than familiarity with a known benchmark.
How accent groups are reported
With Egyptian and Nigerian added, the leaderboard can present a richer accent breakdown in the Private tab. The structure follows Kachru's Three Circles, a classification used to describe different global varieties of English. The project credits Hume AI for the idea used in its leaderboard.
The Open ASR Leaderboard will now report:
- US versus non-US accents, with the non-US group including UK, Australia, Canada, India, Nigeria, and Egypt.
- Inner Circle accents, namely US, UK, Australia, and Canada, versus Outer and Expanding Circle accents, namely India, Nigeria, and Egypt.
As more accents join the set, the project says it will split the Outer and Expanding Circles into separate reporting groups. That change should make the results more useful when a small number of accents currently share the same category.
What Liva AI contributes
Liva AI collects multimodal interaction data through consumer products used by real people. Because Liva owns both the collection environment and the distribution channels, it can design products around specific model gaps across domains, industries, languages, modalities, and interaction types.
That approach gives the evaluation project access to natural, high-context behavior that emerges through real product use. The contrast is important: rather than relying only on artificially elicited or task-based interactions, the private data can reflect how people speak when they are using an application in an ordinary setting. For automatic speech recognition testing, that can expose differences that may not appear in task-based recordings.
What to watch next
The stated direction is to keep improving both representation and trust in automatic speech recognition evaluations. Representation means adding more accents and more languages, while trust means bringing in new private data, new data providers, and methods that reduce the chance of benchmark overfitting.
For readers, the practical takeaway is that future leaderboard rankings may look different from current ones. Systems that perform strongly on broadly public speech tests may need to prove the same strength on Egyptian and Nigerian English, and on other accent groups that are still being added. The Open ASR Leaderboard also signals that the community can continue pushing automatic speech recognition performance worldwide as the private data set expands.
Comments (0)