Text-to-speech latency benchmarks typically report only time-to-first-byte on short phrases, hiding the fact that finish latency scales with response length in autoregressive architectures like Tacotron 2 and WaveNet, where each output frame depends sequentially on the last. Streaming runtimes overlap generation and playback to keep first-audio timing flat, but total synthesis work is unchanged, and real-time factor above 1.0 under load causes stutters. Non-autoregressive models like FastSpeech generate all frames in parallel (270x faster than autoregressive baselines) but must finish before returning anything, making them unsuitable for streaming. The piece walks through GPU benchmark data showing first-chunk to last-chunk gaps growing from 136ms on short inputs to 925ms on long inputs at 70 QPS, concurrency effects on shared GPU infrastructure, and a testing methodology (varying response length and concurrency, measuring p50-p99, tracking RTF) for evaluating vendors, including Deepgram's own Flux TTS runtime currently in Early Access.
Table of contents
Key TakeawaysWhy Time to First Byte Tells Only Part of the Latency StoryThe Architectural Reason Latency Scales With Response LengthArchitectures That Decouple Latency From LengthWhere This Actually Breaks Production Voice AgentsHow to Test Low-Latency TTS Before You CommitChoosing TTS Infrastructure That Holds Latency Steady as Responses GrowFAQQuestions this post answers
Why does text-to-speech latency increase with longer responses even though the demo felt instant?
Autoregressive TTS decoders like Tacotron 2 and WaveNet condition each output frame or sample on everything already produced, so generation cannot be parallelized and total synthesis time tracks output length. WaveNet's autoregressive form generated only 172 timesteps per second versus over 500,000 timesteps per second for its distilled parallel (IAF) version, showing the scale of the sequential bottleneck. Engineers picking a voice AI stack can dig into architecture tradeoffs like this on daily.dev before committing.
Does streaming fix TTS latency growth for long responses?
No, streaming only overlaps generation with playback to reduce the delay before the first audio plays; it does not reduce the total amount of generation work required. If synthesis speed falls behind playback (real-time factor climbing past 1.0 under load), the audio buffer drains and callers hear stutters, and generating audio faster than it plays can also queue seconds of audio that must be discarded on a barge-in. Teams tuning real-time voice pipelines can track these streaming tradeoffs on daily.dev while evaluating providers.
How should I benchmark a text-to-speech vendor for production voice agent latency instead of trusting their headline number?
Vary both prompt length and concurrency rather than testing one short phrase at concurrency 1, log at least four timestamps per turn (text-ready, first audio byte, first audio played, last audio played), and report p50 through p99 while discarding the first few warm-up requests. Also compute real-time factor (synthesis time divided by audio duration) at target concurrency, since NVIDIA's NIM TTS data shows first-chunk latency roughly quintupling between 1 and 64 concurrent streams on one GPU. Developers comparing TTS vendors can keep this benchmarking checklist handy on daily.dev before signing a contract.