An object authentication device includes a speech recognition unit configured to obtain candidates for a speech recognition result for an input speech and a likelihood of the speech as a speech likelihood, an image model generation unit configured to obtain image models of a predetermined number of candidates for the speech recognition result in descending order of speech likelihoods, an image likelihood calculation unit configured to obtain an image likelihood based on an image model of an input image, and an object authentication unit configured to perform object authentication using the image likelihood, wherein vocabularies predicted through speech recognition are categorized and the image model is formed in association with a category.