The present technology relates to an acoustic signal processing apparatus, an acoustic signal processing method, a program, and a recording medium for improving a localization of sound of a sound image at a position deviated from the front center plane of a listener to the left side of the right side. A binauralization processing unit generates a first binaural signal by superimposing an HRTF on an opposite side of a sound source on an acoustic signal and a second binaural signal by attenuating components of a band in which a first notch and a second notch of the HRTF on the opposite side of the sound source appear among components of a signal obtained by superimposing an HRTF on a side of the sound source on the acoustic signal. A crosstalk compensation processing unit performs, with respect to the first binaural signal and the second binaural signal, a crosstalk compensation for canceling out an acoustic transfer characteristic and a crosstalk. The present technology can be applied to, for example, an AV amplifier.