Provided is a signal processing apparatus, including a filter unit that filters an audio signal created by decimating a portion of frequency components by an all-pass filter and outputs a filtering result thereof as improvement components to improve sound quality of the audio signal, and an adder that generates an improved sound in which the sound quality of the audio signal is improved by adding the improvement components to the audio signal.