IBM released two new 470M-parameter English speech recognition models in the Granite Speech family: granite-speech-5.0-470m-turboctc (Apache 2.0) and granite-speech-5.0-470m-turboctc-nc (CC-BY-NC-SA-4.0, trained on more data). Both are encoder-only Conformer-based models achieving over 12,600 RTFx throughput on an NVIDIA H200 GPU, meaning they can transcribe more than 3.5 hours of speech per second with batched inference. Aggregate word error rates on OpenASR Leaderboard public test sets are 4.85% for the noncommercial model and 5.00% for the Apache 2.0 model. Unlike prior Granite Speech models with an LM and LoRA adapters, these drop speech translation and keyword biasing but gain over 20x faster throughput and a smaller footprint, making them suited for edge speech-to-text. The models generate 12.5 tokens per second (down from 50 characters per second in earlier encoders) via three stages of 2x subsampling, and are trained on natural datasets (MLS, YODAS, Common Voice, LibriSpeech, VoxPopuli, AMI, GigaSpeech, SPGISpeech) plus synthetic data generated with gpt-oss models and StyleTTS2. The models are natively supported in a source install of transformers.
Table of contents
TL;DR Performance Model Architecture Training Data Usage TL;DRPerformanceModel ArchitectureTraining DataUsageQuestions this post answers
What throughput does the IBM Granite Speech 5.0 Turbo CTC model achieve on an H200 GPU?
It exceeds 12,600 RTFx on an NVIDIA H200 GPU, meaning it can transcribe more than 3.5 hours of speech in one second using batched inference. The 470M-parameter encoder-only model achieves this while also scoring around 4.85-5.00% aggregate word error rate on OpenASR Leaderboard public test sets, depending on the licensed variant used. Track new speech-model releases like this one on daily.dev as ASR throughput benchmarks evolve.
What is the difference between granite-speech-5.0-470m-turboctc and granite-speech-5.0-470m-turboctc-nc?
The two IBM Granite Speech 5.0 models differ in training data and licensing: the -nc variant trains on additional data and is licensed CC-BY-NC-SA-4.0, while the standard turboctc model trains on a smaller dataset and is Apache 2.0 licensed. The noncommercial model is slightly more accurate overall, with a notable edge on SPGI Speech but a disadvantage on the chunked Earnings22 test. Compare licensing and accuracy trade-offs for models like this on daily.dev before picking one for production.
How do Granite Speech 5.0 models achieve faster transcription than previous Granite Speech encoders?
They operate at 12.5 tokens per second instead of the 50 characters per second used by prior Granite Speech encoders, achieved through three stages of 2x subsampling applied to the 100 frames per second log Mel spectrogram front end. Combined with an encoder-only architecture (dropping the LM and LoRA adapters used previously), this yields over 20x faster throughput and a 470M-parameter footprint suitable for edge devices. Follow architecture-level speedups like this subsampling technique on daily.dev when evaluating ASR models.