Speech recognition of even a speaker who uses a speech recognition system is enabled by using an extended recognition dictionary suited to the speaker without requiring any previous learning using an utterance label corresponding to the speech of the speaker. An extended recognition dictionary learning device includes an utterance variation data calculating section for comparing an acoustic model sequence output from a speech recognition result and an input correct acoustic model sequence to calculate a correspondence between the models as utterance variation data; an utterance variation data classifying section for classifying the calculated utterance variation data into widely appearing utterance variations and unevenly appearing utterance variations; and a recognition dictionary extending section for defining a plurality of utterance variation sets by combining the classified utterance variations and thereby extending the recognition dictionary for each utterance variation set according to the utterance variations included in each utterance variation set. A speech recognition device uses the extended recognition dictionary for each utterance variation set to output a speech recognition result.