The present invention relates to an audio signal processing apparatus and an audio signal processing method which perform binaural rendering.The present invention provides an audio signal processing apparatus which performs binaural filtering on an input audio signal, including: a direction renderer which localizes a direction of a sound source of the input audio signal and a distance renderer which reflects an effect in accordance with a distance between the sound source of the input audio signal and a listener, in which the distance renderer obtains information on a distance (an ipsilateral distance) and an incident angle (an ipsilateral incident angle) of the sound source with respect to an ipsilateral ear of the listener and information on a distance (a contralateral distance) and an incident angle (a contralateral incident angle) of the sound source with respect to a contralateral ear of the listener, determines an ipsilateral distance filter based on at least one of the obtained information of the ipsilateral distance and the ipsilateral incident angle, determines a contralateral distance filter based on at least one of the obtained information of the contralateral distance and the contralateral incident angle, and filters the input audio signal with the determined ipsilateral distance filer and contralateral distance filter, respectively, to generate an ipsilateral output signal and a contralateral output signal.