An audio signal processing apparatus includes a band signal component extraction unit for extracting signal components in a predetermined frequency band from an audio signal input by an audio collection unit for collecting at least an outgoing call audio upon an audio phone call, a stationary signal component extraction unit for extracting a stationary signal component from the signal components, a signal adjustment unit having a level adjustment function of adjusting an output signal level with respect to an input signal level and an input and output characteristics change function of changing input and output characteristics upon level adjustment in the level adjustment function through a control signal and configured to set an incoming call audio signal upon the audio phone call as the input signal, and a control signal generation unit for generating the control signal for changing the input and output characteristics by using the stationary signal component.