A voice processing apparatus includes: a phase difference calculation unit which calculates for each frequency band a phase difference between first and second frequency signals obtained by applying a time-frequency transform to sounds captured by two voice input units; a detection unit which detects a frequency band for which the percentage of the phase difference falling within a first range that the phase difference can take for a specific sound source direction, the percentage being taken over a predetermined number of frames, does not satisfy a condition corresponding to a sound coming from the direction; a range setting unit which sets, for the detected frequency band, a second range by expanding the first range; and a signal correction unit which makes the amplitude of the first and second frequency signals larger when the phase difference falls within the second range than when the phase difference falls outside the second range.