Batch or real-time: choosing the wrong one costs accuracy or costs latency

Teams often pick streaming transcription because it sounds strictly better — the same result, sooner. It is not the same result. Streaming and batch make different trade-offs, and a workload that does not need live output usually pays for streaming in accuracy without getting anything for it.

Why streaming is harder, structurally

A streaming system must emit words before the sentence is finished. That single constraint removes several things a batch system takes for granted.

• **No right-hand context.** Deciding whether someone said "recognise speech" or "wreck a nice beach" is often only possible from what comes next. A streaming model has not heard it yet.
• **Punctuation and casing degrade.** Both depend heavily on sentence structure that has not arrived.
• **Diarization is much harder.** Deciding how many speakers are in a recording, and clustering voices, is far more reliable when the system can consider the whole recording at once. Live, every decision is provisional.
• **Corrections become visible.** Streaming systems revise earlier words as context arrives, which is why live captions visibly rewrite themselves.

None of this makes streaming bad. It makes it a different product, with a ceiling set by the constraint that defines it.

When you genuinely need streaming

The test is simple: **does something have to react before the audio ends?**

• A voice agent deciding what to say next.
• Live captions for accessibility during an event.
• An assistant surfacing information mid-conversation.
• Call-centre monitoring that must intervene during the call.

If someone or something is waiting on the words in order to act, you need streaming, and the accuracy trade is the price of the product working at all.

When batch is quietly the better answer

Far more workloads than teams expect:

• **Recorded meetings and interviews.** Nobody reads the transcript while the meeting is running. It needs to be right, and it needs speakers attributed correctly.
• **Podcast and video production.** Subtitles are edited afterwards regardless.
• **Backlogs.** Thousands of historical files — where throughput and cost dominate and latency is irrelevant.
• **Analytics and QA.** Compliance review, quality scoring, and conversation mining all run after the fact.
• **Search indexing.** The index is built once and queried later.

In all of these, the recording already exists before transcription starts. Streaming buys nothing and costs the accuracy that comes from full-file context.

The middle case people get wrong

The common mistake is a product that *displays* progress live and concludes it needs streaming transcription. Showing a user that their file is 60% processed is a progress problem, not a transcription-mode problem — it needs the API to report progress, not to emit partial words.

It is worth separating the two questions:

• Does the **user** need to see something happening? That is a progress bar.
• Does the **system** need words before the audio ends? That is streaming.

Only the second requires a streaming model, and conflating them costs accuracy on every file.

Practical guidance

If your audio is recorded, use batch, and choose on accuracy — particularly diarization accuracy if more than one person speaks, because speaker attribution errors make a transcript unusable in a way that a slightly wrong word does not.

If your audio is live and something acts on it, use streaming, and evaluate on latency and stability of the partial results rather than headline word error rate.

And if you are unsure, check whether anything in your system would behave differently if the transcript arrived thirty seconds after the audio finished. If the answer is no, the decision has been made for you.

We only build batch, which is a deliberate limitation rather than a stage we intend to grow out of: the whole architecture assumes the file is complete, and that assumption is where the diarization accuracy comes from.