A distance calculation unit (16) obtains the acoustic distance between the feature amount of input speech and each phonetic model. A word search unit (17) performs a word search based on the acoustic distance and a language model including the phoneme and prosodic label of a word, and outputs a word hypothesis and a first score representing the likelihood of the word hypothesis. The word search unit (17) also outputs a vowel interval and its tone label in the input speech, when assuming that the recognition result of the input speech is the word hypothesis. A tone recognition unit (21) outputs a second score representing the likelihood of the tone label output from the word search unit (17) based on a feature amount corresponding to the vowel interval output from the word search unit (17). A rescore unit (22) corrects the first score of the word hypothesis output from the word search unit (17) using the second score output from the tone recognition unit (21). This allows to raise the speech recognition accuracy for tone speech.