Speech recognition

Speech recognition turns an acoustic waveform into a sequence of words or executable commands. A microphone captures pressure changes, after which the system extracts representations such as spectrograms and estimates which phonetic or subword units best match the sound. A language model then favors plausible word sequences, helping distinguish similar pronunciations such as “recognize speech” and “wreck a nice beach.”¹

Traditional systems separated an acoustic model, pronunciation lexicon, and language model; contemporary systems often learn much of this mapping jointly with deep neural networks. The final stage may add punctuation, capitalization, speaker labels, or command interpretation. Recognition is different from speaker recognition: the former asks what was said, while the latter asks who spoke.

Modern accuracy emerged from replacing hand-engineered pattern matching with data-intensive statistical learning. Early commercial systems used hidden Markov models and Gaussian mixture models, while deep neural networks improved acoustic modeling by learning richer representations from large labeled speech corpora.²

Sequence-to-sequence and connectionist temporal classification methods reduced the need for an explicit pronunciation dictionary. Attention-based models can map long stretches of sound directly to characters or subword units, and transformer architectures extend this approach across languages and tasks. Large self-supervised models first learn from unlabeled audio and are then adapted for transcription, translation, or domain-specific vocabulary. Such systems can be highly capable without being equally reliable in every language or speaking situation.

Speech-recognition performance is commonly reported with word error rate, calculated from substitutions, deletions, and insertions relative to a reference transcript. NIST evaluations have helped standardize comparisons, but a single score can conceal important differences between accents, dialects, recording conditions, and conversational styles.

Noise, overlapping speakers, reverberation, code-switching, uncommon names, and domain terminology remain difficult. Accuracy disparities have been measured across demographic groups; one widely cited study found substantially higher error rates for Black speakers than for white speakers in several commercial systems.³ Human review is therefore still valuable for legal, medical, educational, and archival transcripts. Recognition output should be treated as an inference, not a verbatim record guaranteed to preserve meaning.

Speech recognition has important uses beyond dictation and voice assistants. It supports live captions, searchable archives, call-center quality analysis, language documentation, accessibility tools, and interfaces for people who cannot use conventional keyboards. Assistive technology can combine recognition with text-to-speech, switch control, or environmental control, although performance depends on the user’s speech and the surrounding acoustics.

Some systems recognize commands without sending continuous audio to a server, using an on-device wake-word detector or compact local model; others transmit recordings for cloud processing, creating privacy and retention questions. Multilingual models can share representations across languages, but low-resource languages may remain poorly served because training data, orthographies, and evaluation transcripts are scarce. Whisper demonstrated the value of very large, weakly supervised multilingual training while also documenting persistent errors involving names, background sounds, and rare languages.⁴

Word error rate

A transcription metric based on substitutions, deletions, and insertions divided by the number of words in a reference transcript.

Acoustic model

A model that estimates how recorded sound corresponds to phonetic, character, or subword units.

Language model

A model that assigns probabilities to sequences of words or tokens and helps select linguistically plausible output.

Speaker diarization

The process of determining who spoke when in an audio recording.

Wake word

A spoken trigger that activates a voice-controlled device or recognition service.