An apparatus and method of voice activity detection (VAD) are disclosed. To detect a target sound, a target sound detecting apparatus calculates the phase difference corresponding to each frequency component from a plurality of sound signals converted in frequency domain, calculates a characteristic value indicating possibility that the phase difference of the frequency component is within an allowable phase difference range of target sound calculated based on a direction angle of the target sound, and detects the presence and/or absence of target sound using the characteristic value.