A voice processing device includes a voice pitch converting unit that performs a voice pitch converting process with respect to an input voice signal and converts voice pitch of the input voice signal, an error detecting unit that detects an error between the number of samples of an output voice signal, which is expected, and the number of samples of the output voice signal, which is actually output, and a time length control unit that controls adjustment of the time length in such a manner that the time length of the output voice signal is corrected by the amount of the error.