For each of 30 test items, evaluators heard four recordings of the identical Arabic text, spoken in a matched gender voice: one from Munsit, one from each of two commercial TTS systems, and one from a professional human voice talent. 

System identity was hidden, and the order of the four recordings was re-randomized on every single item, so no listener could learn to spot Munsit by position. Each recording was scored independently on its own merits, there was no forced "pick a winner" step. The 30 items were split 10 per dialect (Modern Standard Arabic, Saudi Arabic, Emirati Arabic) and covered two use cases: conversational-assistant style and documentary-style narration.
‍

Who rated the recordings
‍

• 352 unique native Arabic speakers took part, each rating only the dialect they natively speak (251 for MSA, 76 Saudi natives, 25 Emiratis).
• Sessions totaled 369, producing 3,602 valid evaluations after quality screening.
• Every session included a hidden repeated item; anyone who rated it inconsistently was excluded, following ITU-T P.808 crowdsourced evaluation practice. A separate check removed 49 duplicate submissions.
• Pairwise agreement on which system was top-rated was 39.9% overall, well above the 25% you'd expect if all four systems tied by chance.
  ‍

Full Results: Quality Scores vs. "Top-Rated Share"

Once Arabic text-to-speech systems get good, mean-opinion-score (MOS) ratings compress near the top of the 1–5 scale,  a known limitation of standard MOS testing (Kirkland et al., 2023). That's exactly what happened here: Munsit and the human reference are statistically tied on the 1–5 scale, yet Munsit was still the single highest-rated recording in over half of all evaluations. 

The two metrics aren't contradictory, the average smooths out item-by-item differences that the top-rated share preserves. 

‍

‍

Munsit and the human reference differ by no more than 0.04 points on any of the three sub-criteria. Both commercial systems trail by a clearly larger margin, 0.3–0.4 points for the closer one, 1.7–1.8 points for the other.

Results by Dialect: MSA, Saudi, and Emirati
‍

Aggregate numbers can hide Arabic dialect-specific weaknesses, so the benchmark broke results out by MSA, Saudi Arabic, and Emirati Arabic separately. The pattern holds in the two larger samples but softens in the smallest one, for reasons worth understanding before drawing conclusions: 

‍

‍

Why the Emirati result isn't conclusive yet
‍

Munsit still led numerically in Emirati Arabic, but with only 32 sessions the confidence interval on the gap (−14.6 to +30.6 percentage points) spans zero, meaning the sample simply wasn't large enough to rule out chance. This is a statistical power problem, not a tie: the same-size gap seen in MSA and Saudi would need roughly 100 Emirati sessions to reach the same confidence, which is why CNTXT AI has committed to expanding that panel. If you're evaluating a vendor's Gulf-dialect claims, always ask for the sample size behind the percentage, not just the percentage itself

It's a fair question to be skeptical of, so it's worth being precise about what the data actually shows and doesn't. The human reference wasn't rated poorly, 4.30 out of 5 is a strong score by any TTS benchmarking standard, and it's the highest individual quality score in the study. What changed is how often it was the single best option in a four-way, blind, randomized comparison against a system trained specifically on curated dialectal Arabic speech data.
‍

This isn't an isolated finding. Independent industry benchmarks such as TTS Arena and recent academic work on "human fooling rates" (Srinivasavaradhan et al., presented at Interspeech 2025) report a similar pattern: top-tier Arabic text-to-speech systems are increasingly preferred over recordings of the same content in blind tests, largely because synthesis delivers consistent prosody without the fatigue or session-to-session variance that affects even skilled voice talent across long recording runs. 
‍

The honest reading of this benchmark is not "AI beats humans" as a general claim, it's that Munsit has reached the human-quality range for the dialects and content types tested here, on a specific comparison against one set of professional recordings.

Munsit’s strong performance isn’t simply about AI sounding “better” than humans. A major factor is the depth and curation of Arabic dialectal speech data behind the model.
‍

• Dialect-specific data matters: MSA has relatively abundant speech resources, while Saudi and Emirati Arabic remain more challenging for general-purpose TTS systems.
  
  
• Purpose-built Arabic data: Munsit is trained on Arabic speech data built and labelled by CNTXT AI, rather than relying primarily on third-party licensed datasets.
  
  
• Consistent dialect fidelity: This helps Munsit maintain strong performance across pronunciation, naturalness, and dialect fidelity in all three tested varieties.
  
  
• Narrower human-AI gap: The difference was smallest on formal MSA, while Munsit continued to perform strongly on dialectal content.

The takeaway: The blind-test result isn’t evidence that AI is universally better than human voices. It shows what becomes possible when dialect-specific data curation is treated as a core engineering priority rather than an afterthought.

If you're choosing a TTS system for an Arabic-language product, an IVR system, an audiobook pipeline, a customer-support voicebot, benchmark numbers like these are a starting point, not a purchase decision on their own. Based on what this methodology gets right and where it's limited, here's what's worth checking with any vendor:
‍

• Ask whether the test was blind and randomized. Non-blind demos (where you know which clip is the AI) are far more prone to bias than the setup used here.
  
  
• Ask for the sample size behind any dialect-specific claim. A 40%+ win rate on 32 sessions and the same win rate on 250+ sessions are not equally trustworthy.
  
  
• Test on your own scripts and your own dialect, not just the vendor's benchmark items, production content (names, numbers, code-switching, domain jargon) often behaves differently than curated test sentences.
  
  
• Ask which scale is being reported. A 1–5 mean score and a "top-rated share" measure different things; a vendor citing only the flattering one is worth a follow-up question.
  
  
• Check the collection date on any competitor comparison. TTS systems update frequently; a comparison from a specific month can be outdated within a quarter.

The key takeaway isn’t simply that Munsit beat human recordings. In a blind test with 352 native Arabic speakers across three dialects, listeners struggled to distinguish Munsit from professional studio voices, and selected it as the best recording more often than any other option.

The results were strongest for MSA and Saudi Arabic, while Emirati Arabic needs further validation. Ultimately, the best test is your own: evaluate Munsit using your scripts, target dialect, and real-world use case.

Want to see how this plays out on your own content? Send 20 of your real Arabic scripts and get a Munsit benchmark run against your dialects, customer journeys, and channels. Test Munsit on your scripts.