State-of-the-art speech recognition models like Whisper, Deepgram, and Phi-4 achieve near-human accuracy on standard benchmarks but fail 39% of the time when transcribing street names spoken by linguistically diverse users. Research from Together AI introduces two new benchmarks (SF Streets and US Streets) to expose this gap, revealing an 18% accuracy disparity between non-English and English primary speakers. To address this, researchers developed a 'cross-lingual style transfer' technique using the open-source XTTS model to generate synthetic training data — forcing TTS models to apply non-English phonetics to English street names. Fine-tuning Whisper-Base on fewer than 1,000 such synthetic samples yielded a 60% relative accuracy improvement, with the largest gains for non-English speakers. The SF Streets dataset and US Streets benchmark will be publicly released to help the community move beyond aggregate metrics.
