A signal processing apparatus, for processing sounds collected in an environment where a target sound and an interfering sound are mixed in order to estimate a diffuse interfering sound accurately, is provided. The signal processing apparatus includes phase difference calculating means and generating means. The phase difference calculating means calculates a phase difference between the first input signal and the second input signal. The first input signal is generated based on the first input sound which is input in the environment where the target sound and the interfering sound are mixed. The second input signal is generated based on the second input sound which is input in the environment. The generating means generates an estimated interfering sound signal, based on the phase difference and the first input signal.