Speech synthesis

Speech synthesis produces a speech waveform from a symbolic or linguistic representation. A typical text-to-speech system first normalizes written input, expands abbreviations and numbers, predicts pronunciation, assigns prosodic structure, and then generates audio.¹ The final signal must represent segmental information—consonants and vowels—as well as suprasegmental features such as stress, rhythm, phrasing, and intonation.

Traditional systems separated linguistic analysis from an acoustic synthesizer. Formant synthesizers generated speech from models of resonant frequencies in the vocal tract, while concatenative systems joined recorded units such as diphones or syllables. These approaches offered explicit control but could sound mechanical or produce audible joins when the requested context differed from the recorded material.²

Synthesis differs from speech recognition: recognition maps sound to language, whereas synthesis maps language or control instructions to sound. Some systems also accept phonemes, musical notation, or real-time articulatory parameters rather than ordinary text.

The field developed from electromechanical and analog speech machines into software-controlled digital systems. Homer Dudley’s Voder, demonstrated publicly in 1939, showed that an operator could control an artificial voice; later work made synthesis programmable and increasingly independent of manual performance.¹ Dennis Klatt’s formant-synthesis research helped establish detailed computational models for English pronunciation and intonation.²

Statistical parametric synthesis later represented speech with learned parameters, enabling compact voices and flexible control. Neural systems changed the dominant design: WaveNet generated audio samples with a deep neural model, while Tacotron learned a sequence-to-sequence mapping from text to acoustic features before waveform reconstruction.⁴⁵

Contemporary systems often combine linguistic front ends with neural acoustic and vocoder components. The result can be more natural, but quality still depends on training data, pronunciation coverage, latency, computational cost, and safeguards against deceptive or unauthorized voice imitation.

Naturalness depends as much on prosody as on correct phonemes. A synthesizer must decide where a phrase begins and ends, which words receive emphasis, how questions differ from statements, and how speech rate and pitch change across an utterance. Speech Synthesis Markup Language (SSML) provides standardized controls for pronunciation, pauses, emphasis, rate, pitch, and voice selection.³

Applications include screen readers, navigation systems, telecommunications, language learning, public announcements, embedded devices, and conversational interfaces. For people who cannot use their natural voice, personalized synthesis can preserve a familiar vocal identity when recordings are collected before medical treatment, although the process raises practical and ethical questions about consent, storage, and later control of the voice.

Evaluation commonly separates intelligibility from naturalness and speaker similarity. Listening tests such as mean-opinion-score procedures remain widely used, but objective signal measures alone do not fully capture prosodic appropriateness, conversational timing, or whether a voice sounds trustworthy in a particular setting.⁶

Speech synthesis is not limited to reading ordinary text. Systems may render phonetic transcriptions directly, produce speech in low-resource languages with limited recordings, or use restricted vocabularies in devices where predictable pronunciation matters more than open-ended fluency. Concatenative systems can still be advantageous in such settings because their output is stable and auditable.

A less visible technical problem is the boundary between words and speech units. Coarticulation means that a sound changes according to its neighbors, so simply assembling idealized isolated phonemes often creates unnatural transitions. Prosody also carries information that is not explicit in spelling, including irony, politeness, uncertainty, and turn-taking cues.

Neural synthesis introduces a different long tail of concerns. A model may reproduce recording artifacts, encode demographic biases in its training data, or generate convincing speech with incorrect factual content. Voice conversion and cloning can support accessibility and creative work, but they also make provenance, watermarking, disclosure, and authorization central engineering requirements rather than optional features.

Formant synthesis

Speech generation from modeled resonances and source signals rather than directly replayed recordings.

Concatenative synthesis

Speech generation by joining recorded units such as phones, diphones, syllables, or words.

Prosody

The rhythm, stress, timing, pitch, and phrasing patterns that organize spoken language.

Vocoder

A system that converts acoustic or learned features into an audible speech waveform.

SSML

Speech Synthesis Markup Language, an XML-based standard for controlling synthesized speech output.