A voice processing device includes a zone detection unit which detects a voice zone including a voice signal or a non-steady sound zone including a non-steady signal other than the voice signal from an input signal and a filter calculation unit that calculates a filter coefficient for maintaining the quality of the voice signal in the voice zone while suppressing the non-steady signal in the non-steady sound zone according to the detection result by the zone detection unit, in which the filter calculation unit calculates the filter coefficient by using a filter coefficient calculated in the non-steady sound zone for the voice zone and using a filter coefficient calculated in the voice zone for the non-steady sound zone. In one embodiment, a verification unit verifies a constraint condition of the filter coefficient based on whether the amount of suppression of the non-steady sound signal that would result from applying the filter to the sound signal is less than or equal to a threshold value.