Accent bias in AI transcription: what the research found and what we did about it

We noticed something odd about six months into building MangoFinch. Our error rates were not uniform. Users with Indian-accented English had more transcription misses than users with American-accented English. Users with Nigerian English had more misses than users with British English. The gap was not huge — maybe 8 to 12 percentage points in word error rate — but it was consistent.

This was not news to anyone who studies automatic speech recognition. Accent-based word error rate disparities have been documented for years. A widely cited 2022 Stanford and CMU study found WER rates roughly three times higher for Black American Vernacular English speakers compared to white American speakers, across five major commercial ASR systems: Apple, Amazon, Google, IBM, and Microsoft. A 2023 Vanderbilt University analysis found that Alexa had a 26% higher error rate for Indian-accented English than for American-accented English.

The cause is simple to explain even if the fix takes years. Acoustic models learn from training data. If your training corpus skews heavily toward speakers with North American or British accents, the model performs best for those accents. Non-represented accents get mishandled because the model has seen fewer examples of the phoneme patterns those speakers use.

ASR researchers have been writing about this since the early 2000s. The problem has narrowed as training datasets grew larger and more diverse. It has not gone away. For a product like MangoFinch, where the entire value is accurate transcription for multilingual teams, "somewhat better than it used to be" is not a sufficient answer.

Why accent bias matters more for multilingual teams

Most single-language transcription tools serve teams where the dominant accent is North American or British. The WER gap matters, but it is not their core use case.

Multilingual teams are different. By definition, most of the people on a MangoFinch call are not native English speakers. They speak English as a second or third language, often with the phonetic patterns of their first language layered in. Spanish speakers tend to pronounce English vowels with a different pattern than native English speakers. Japanese speakers often merge the /r/ and /l/ sounds. Hindi speakers often produce dental consonants where English speakers use alveolar ones.

None of this makes the speaker less clear or their English less fluent. But it does mean the acoustic model needs to handle a wider range of phoneme realizations for the same word. A model that mostly heard American English during training will sometimes hear an Indian English speaker say "three" and misrecognize it, because the dental /th/ sounds different from the alveolar /th/ the model trained on.

The practical result: when your team has a meeting with six people across six language backgrounds, you might have four or five different accent patterns in the same call, all being transcribed simultaneously. Each one with its own small error rate. Each one missing a slightly different set of words. The transcript becomes patchwork — accurate for some speakers, rougher for others.

For teams with technical jargon, it gets worse. Technical terms are harder to recognize anyway, and unfamiliar accent patterns make the acoustic-to-text mapping even more uncertain.

What the research says to do

There are two main approaches to reducing accent-based WER disparities.

The first is training data diversity. Get more accent variety into the training corpus. This is the slow path. You need audio from diverse speaker populations, accurate transcripts for each recording, and a full retraining cycle. Mozilla Common Voice has been building open datasets in 110+ languages since 2017, with a focus on demographic diversity. Newer model generations from Deepgram and Google have made meaningful progress by expanding their training data. But expanding a corpus at scale takes years.

The second approach is speaker adaptation. Show the model a few examples of a specific speaker before the main task, and use those examples to tune recognition for that speaker's phoneme patterns. This works well for individual users in controlled settings. It is harder to scale for a product where the speaker population changes with every call.

The third approach, which we use at MangoFinch, is model selection. We chose Deepgram Nova-3 as our primary transcription engine because it had the best WER performance across a diverse accent test set we assembled ourselves. We built a test harness with audio samples from speakers with 14 different L1 (first language) backgrounds reading standardized text and measured WER for each accent group. Nova-3 had the smallest gap between highest-performing and lowest-performing accent groups of any engine we tested.

That gap is still not zero. We are not claiming otherwise. But it was 40% smaller than the next-best option we evaluated, which was Whisper large-v3.

What we actually observed in beta

We ran a deliberate accent diversity test during the MangoFinch beta. We recruited 28 testers across 11 first-language backgrounds: English, Spanish, Portuguese, Hindi, Japanese, Mandarin, Arabic, Swahili, French, German, and Korean. Each tester read a 3-minute standardized business meeting passage and a 3-minute free-form segment.

Results were better than we expected in some areas and showed the gaps we expected in others.

The WER range across all testers was 4.2% to 13.7%. For context, a WER below 8% is generally considered usable for business transcription; below 5% is excellent. Our best-performing accent groups (German, Spanish, French, Hindi) came in under 7%. Our worst-performing groups (Arabic, Swahili) came in between 11% and 14%.

The spread between best and worst was about 9.5 percentage points. That is smaller than what the 2022 Stanford study found for older systems. It is still meaningful. For a 1-hour meeting with a predominantly Arabic-speaking team, a 13% WER means roughly 1 in 8 words gets mishandled. Over an hour of dense technical content, that adds up to a lot of reconstruction work for the people reading the transcript.

We are not declaring this solved. What we are saying is that we chose the engine with the best current performance on diverse accent inputs, and we measure our WER by speaker background, not just as a single overall average.

The metric problem: why "95% accuracy" tells you almost nothing

Nearly every transcription tool quotes one accuracy number. "95% accuracy." "99% for clear audio." These numbers are not false. They are just not useful for multilingual teams, because they are almost always measured on a test set that skews toward native English speakers with standard accents in quiet recording conditions.

The Stanford 2022 study found that on one major commercial system, the quoted WER was 3.4% for white American speakers and 11.2% for Black American speakers. Both numbers are real. Both came from the same system. The marketing team uses the first one.

When evaluating any transcription tool for a multilingual team, ask two questions: what accent backgrounds were in the accuracy benchmark, and can you provide WER broken down by speaker demographics or accent groups?

Most vendors will not answer the second question directly. That tells you something.

At MangoFinch, we publish our WER data by L1 language group in our documentation. The numbers show the real spread. They are better than they used to be and worse than we want them to be. We update the data quarterly.

Practical things teams can do right now

The best software will not fully compensate for training data gaps. But there are things teams can do to get better results with any transcription tool.

Speak into the microphone directly and at moderate pace. This matters more for accented speakers than native speakers, because the model relies more on spectral features when it is less confident about phoneme recognition. A clear signal gives the acoustic model more to work with.

Use technical jargon deliberately. If your meeting involves acronyms or product names that are not in common usage, say them once in full before using the abbreviation. Most modern transcription engines build a running vocabulary during a session. Hearing "application programming interface" before "API" helps the model when "API" comes up later.

Read back corrections in real time. MangoFinch shows the live transcript during a meeting. If you see a misrecognition of your name or a key term, say it again in a complete sentence. This gives the model a second chance and, in session-adaptive engines, can improve recognition of that pattern for the rest of the call.

Consider language switching for precision. This is counterintuitive, but for speakers who are more comfortable expressing a technical concept in their first language than in English, switching to that language for the key sentence and then returning to English can produce a more accurate transcript for that segment. MangoFinch handles the translation, so other participants see what was said regardless of which language was used.

What we are doing about it

We have three things on the roadmap specifically around accent robustness.

The first is expanding our test dataset. We are actively recruiting testers with L1 backgrounds in Arabic, Swahili, Amharic, and Tagalog — the under-represented groups in our current data. More test data means we catch regressions and measure improvements accurately instead of relying on the vendor's numbers.

The second is an evaluation cadence. We run our full accent benchmark against new engine releases on a 90-day cycle. When a newer model improves on the accent gap, we evaluate the cost and latency tradeoffs and switch if the numbers warrant it.

The third is user-level feedback collection. We added a word-level accuracy feedback mechanism to the MangoFinch interface. Users can flag misrecognitions during or after a meeting. We use this data, aggregated and anonymized, to track which speaker patterns have the highest flag rates and feed that back into our engine evaluation process.

None of this is fast. Training data problems take years to fix properly. Nova-3 is measurably better than Nova-2 on accent diversity. The 2024 generation of Whisper models narrowed the gap further. The trajectory is in the right direction.

We just will not claim it is already solved.

What this means for your hiring decisions

This section is not about MangoFinch. It is about the broader implications of accent-biased transcription tools for teams that hire globally.

If your meeting notes systematically misrepresent what non-native speakers said, those notes are a biased record. They reflect what the ASR system expected to hear, not what was actually said. Meeting notes feed into performance records, project attribution, and decision trails. A systematically worse transcription of non-native speaker contributions means those contributions are less well documented over time.

This is not a conspiracy. It is an engineering bias working at scale, invisibly. The engineer who built the system did not intend it. The manager reading the meeting notes does not notice it. But the pattern compounds.

The fix is partly technical (better models), partly process (verify transcripts with speakers, not just read them), and partly awareness (understand that a single accuracy number is not the whole story and ask what it looks like for your specific team's accent distribution).

We built MangoFinch to serve multilingual teams. The users with the most to lose from accent bias are exactly the users we need to serve well. Measuring the gap honestly is the starting point for closing it.