In a text-to-speech (TTS) system, a database including sample speech units for unit selection may be include both units represented by sample audio segments as well as parametric representations of units created by Hidden Markov Models (HMMs). Inclusion of parametric representations in the database may reduce the storage necessary to maintain the database. The parametric representations may be configured to match a voice of the audio segments. The parametric representations may correspond to phonetic units that are less frequently encountered in TTS processing, such as rare diphones or phonemes corresponding to foreign languages. Multiple foreign language HMM models may be used to enable polyglot synthesis with a reduction in storage capacity requirements. Parametrically stored speech units may be combined with speech segments generated during processing time by a parametric model.