Transcription accuracy is a lie — here’s what actually matters

Every transcription company advertises accuracy. "95% accurate." "Human-level transcription." "99% for clear audio." These numbers sound impressive. They are also close to meaningless for anyone who actually relies on transcripts to do their job.

I want to make an argument that the transcription industry is measuring the wrong thing, and that this measurement problem causes real harm to users who trust these numbers.

What WER actually measures

Word Error Rate is the standard metric. The formula is simple: take a reference transcript (the "correct" version, usually produced by a human), compare it to the machine output, count the insertions, deletions, and substitutions, divide by total words in the reference.

A 5% WER means the system got 95 out of every 100 words right. That sounds great.

Here is what it looks like in practice. A 60-minute meeting contains approximately 8,000-9,000 words (average speaking rate is 130-150 words per minute, and not everyone talks the whole time). At 5% WER, that is 400-450 wrong words. Spread across a meeting, that is roughly 7 wrong words per minute.

Seven wrong words per minute. In a meeting about quarterly revenue, hiring decisions, or product launch dates.

Not all errors are equal

This is where WER falls apart as a useful metric. It treats every word equally. Getting "the" wrong counts the same as getting "million" wrong. Transcribing "we should" as "we could" is one substitution error, same as transcribing "uh" as "um."

But these are not the same. They are not even close.

Consider two transcripts of the same sentence. The speaker said: "We committed to shipping 14 units to the Berlin warehouse by March 15th."

Transcript A: "We committed to shipping 14 units to the Berlin warehouse by March 15th." (0% WER)

Transcript B: "We committed to shipping 40 units to the Berlin warehouse by March 5th." (2 errors out of 13 words = 15% WER)

Transcript C: "We, um, committed to, uh, shipping 14 units to the Berlin warehouse by March 15th." (4 insertions out of 13 words = 31% WER)

Transcript B has a lower WER than Transcript C. But Transcript B is catastrophically wrong. Someone acting on Transcript B would ship the wrong quantity to the wrong deadline. Transcript C is perfectly usable. It just has filler words.

WER says C is three times worse than B. Any human reading these transcripts would say the opposite.

The proper noun problem

Names break transcription in ways that WER dramatically underweights.

In a recent test, we ran 50 meeting recordings through three different transcription engines and manually reviewed the output. The WER scores were all in the 4-7% range. Respectable numbers.

Then we looked specifically at proper nouns: company names, people's names, product names, and place names. The error rate on proper nouns was 23-31% across all three engines. Nearly one in three names was wrong.

This is a disaster that WER hides. If a transcript says "Johnson approved the Meridian contract" but the speaker actually said "Jensen approved the Meridian contract," that is one substitution error out of maybe 10,000 words. WER barely notices. But for anyone trying to figure out who approved what, the transcript is lying to them.

The same applies to numbers. "Revenue was 4.2 million" versus "Revenue was 4.8 million" is a single-character error that WER scores as one substitution. The difference is $600,000.

How errors compound in downstream tasks

Wrong words do not stay contained in the transcript. They propagate.

Meeting summaries generated from transcripts inherit errors. If the transcript says "we decided to delay the launch" but the speaker said "we decided to relay the launch plan," the summary will report a delay that never happened. The AI summarizer has no way to know the transcript is wrong. It faithfully summarizes fiction.

Action items extracted from transcripts are worse. "John will send the report to the Paris office" versus "John will send the report to the Prague office" produces an action item that sends the wrong deliverable to the wrong city. One substitution error, one misrouted document.

Search is affected too. If someone searches the transcript archive for "Berlin warehouse" and the transcript says "Burlington warehouse," that search returns nothing. The information exists but is unfindable because of a transcription error that WER barely registers.

Calendar integrations break in a similar way. Several tools now auto-create follow-up events from action items in transcripts. "Schedule a review for April 3rd" transcribed as "April 30th" creates a calendar event four weeks late. The person who relies on that auto-created event has no reason to verify the date. They trust the system. The system got one digit wrong.

The most insidious propagation happens in compliance contexts. Legal and financial teams transcribe meetings for regulatory requirements. A pharmaceutical company needs accurate records of what was discussed in clinical trial reviews. An investment bank needs transcripts of trading desk communications. In these environments, a transcription error is not just inconvenient. It is a compliance risk. And WER does not distinguish between errors that are merely annoying and errors that trigger regulatory violations.

We measured this propagation effect across 200 meeting transcripts. For every 1% increase in WER, downstream task accuracy (measured by whether summaries, action items, and search results were correct) dropped by 2.3%. Errors do not have a linear impact. They multiply.

What we measure instead

At MangoFinch, we still calculate WER because it is the industry standard and people expect to see it. Our current WER for English-only meetings is in the low single digits. For multilingual meetings with code-switching, it roughly doubles. Those numbers are fine. They are also insufficient.

We track four additional metrics that we think matter more.

**Entity accuracy.** We separately score how well the transcription captures names, numbers, dates, and technical terms. We maintain a per-meeting entity list (built from calendar invites, participant names, and previous meeting context) and check each entity against the transcript. Current entity accuracy: 87% for names, 91% for numbers, 94% for dates.

The gap between our 87% name accuracy and our 4.1% WER tells the story. The overall WER looks good. The names are wrong 13% of the time. For a tool people use to track who said what, 13% name errors is the number that matters.

We break entity accuracy down further by category. Personal names are the hardest: 87%. Company names and product names are slightly better at 89%, likely because they appear more frequently and the model has more training data. Phone numbers and email addresses are at 93% because they follow predictable patterns. Dates and times are at 94%. Dollar amounts and percentages: 91%.

The per-entity breakdown lets us target improvements. When we added custom vocabulary loading (users can upload a participant list before a meeting), name accuracy jumped from 79% to 87%. That single feature, which took two weeks to build, moved the needle more than three months of general model tuning.

**Semantic accuracy.** Does the transcript preserve the meaning of what was said? This is harder to measure automatically, so we use a two-layer approach. First, we run the original and transcribed text through a sentence embedding model and compute cosine similarity. Segments scoring below 0.85 similarity get flagged. Second, we do a weekly human review of 100 randomly sampled flagged segments.

Current semantic accuracy: 96.2% of segments score above 0.85 similarity. The 3.8% that fall below are almost always segments with heavy code-switching, background noise, or overlapping speakers.

**Action item accuracy.** We extract action items from transcripts and compare them against action items extracted from the reference (human-verified) transcript. An action item is correct if it has the right person, the right task, and the right deadline. Getting two out of three right does not count.

Current action item accuracy: 82%. This is the number I lose sleep over. One in five action items has something wrong. Usually it is the person's name (back to the entity accuracy problem) or a date ("next Friday" getting resolved to the wrong calendar date).

**Translation fidelity.** Since MangoFinch does real-time translation, we also measure whether the translated version preserves the meaning of the original. We use back-translation (translate the translation back to the source language) and compare semantic similarity. Current translation fidelity: 91% for European language pairs, 84% for Asian-European pairs.

Why the industry resists better metrics

WER is easy to calculate, easy to compare, and easy to put on a marketing page. "95% accurate" is clean. "87% entity accuracy with 82% action item accuracy and 96.2% semantic preservation" is not a tagline.

There is also a benchmark problem. WER has standard test datasets: LibriSpeech, Switchboard, CallHome. Everyone tests against the same recordings and publishes comparable numbers. No standard dataset exists for entity accuracy or action item accuracy in business meetings. Building one requires thousands of hours of annotated meeting audio, and no one has published that dataset yet.

We are working on releasing our internal benchmark dataset (anonymized, of course) by Q4 2026. It will include 500 hours of multilingual meeting audio with entity-level annotations, semantic accuracy scores, and extracted action items. If other companies start measuring against it, we all get better. If we are the only ones using it, at least our numbers mean something.

There is also a marketing incentive problem. WER is a number that only goes up. Every year, models get better, and every company can report a higher accuracy percentage. Switching to entity-specific metrics would mean some companies have to report lower numbers than their current WER. No marketing team wants to replace "95% accurate" with "87% name accuracy." Even if the second number is more honest and more useful, it looks worse on a landing page.

This is the same dynamic that kept the mobile phone industry reporting peak download speeds instead of real-world throughput for years. The misleading number is always the more flattering one.

What this means for buyers

If you are evaluating transcription tools for your organization, here is what I would actually ask during a vendor demo.

Do not ask: "What is your accuracy rate?" You will get a WER number that tells you almost nothing about real-world performance.

Ask: "What is your accuracy on proper nouns?" If they cannot answer this, they are not measuring it. That means they do not know, which means it is probably bad.

Ask: "Can you run a test on a recording of our actual meetings?" Generic benchmark audio (clean studio recordings of people reading scripts) produces dramatically better scores than real meeting audio with crosstalk, background noise, accents, and domain-specific jargon.

Ask: "What happens to accuracy when speakers switch languages?" If the answer is "we only support one language at a time," that is not a multilingual tool. It is a monolingual tool with a language dropdown.

Ask: "Show me a transcript with errors, not a perfect one." Any vendor can cherry-pick a clean segment. Ask to see a segment where the transcription struggled. How they handle errors tells you more about the product than how they handle easy audio.

Ask: "How does accuracy change over the course of an hour-long meeting?" Transcription engines can degrade as meetings get longer. Speaker fatigue changes speech patterns. Background noise accumulates. Some engines handle long-duration audio worse than short segments. If the vendor only benchmarks 5-minute clips, they are hiding long-form performance.

WER gets worse with multiple languages

Everything I have described so far applies to monolingual meetings. For multilingual meetings, WER is even more misleading.

Standard WER compares against a single reference transcript. But what is the reference for a meeting where someone says a sentence that is half English and half Japanese? The reference transcript needs to capture the language switch, the original text in both languages, and optionally a translation. Most WER benchmark datasets do not include multilingual audio at all.

When we calculate WER for multilingual meetings, we first segment by language, then calculate WER per language, then weight by segment duration. Our English WER is 4.1%. Japanese is 6.3%. Spanish is 5.2%. Portuguese is 5.8%. But the combined WER for a meeting that switches between all four is 7.8%, which is higher than any individual language. The extra errors come from the language boundaries themselves, where the engine is deciding which language it is hearing.

A vendor quoting a single WER number for a "multilingual" product is either measuring monolingual segments and averaging them (which hides boundary errors) or not measuring multilingual performance at all.

The uncomfortable truth

Every transcription tool, including MangoFinch, gets things wrong. The question is not whether errors happen. It is whether you can see where they happen and whether the errors that slip through are the ones that matter least.

WER hides this by averaging everything into a single number. A transcript can score 95% on WER while getting every number and every name wrong, as long as it nails the articles, prepositions, and filler words that make up the bulk of spoken language.

We think the industry needs to retire WER as the primary quality metric and move to a weighted, task-specific scoring system. Names should weigh more than articles. Numbers should weigh more than conjunctions. Meaning preservation should weigh more than word-for-word matching.

We are not there yet industry-wide. But we publish our entity accuracy, semantic accuracy, and action item accuracy numbers on our status page, updated weekly. If those numbers embarrass us, good. That is how they get better.