A voice processing device includes a voice receiver that receives a voice, an imager, an image acquirer that acquires a captured image captured by the imager, an utterer identifier that identifies an utterer based on the voice received by the voice receiver and the captured image acquired by the image acquirer, a voice determiner that determines whether the voice is a specific word based on the voice received by the voice receiver and an image of the utterer identified by the utterer identifier, the image being included in the captured image, and a voice transmitter that switches a transmission destination of the voice received by the voice receiver based on a determination result by the voice determiner.