A voice processing device includes a first sound collecting unit for generating a first voice signal; a human-body vibration obtaining unit for generating a human-body vibration signal; a first distance calculating unit for calculating a ratio in power or amplitude between the human-body vibration signal and the first voice signal and for calculating a first distance from the first sound collecting unit to a sound source in accordance with the ratio and distance estimation information; a second distance calculating unit for calculating, for each of a plurality of frequencies, a second distance from the first sound collecting unit to a sound source which produces a component of a frequency of a first frequency signal; a gain determining unit for determining, for each of the plurality of frequencies, a gain based on a comparison result between the first distance and the second distance.