In a signal processing apparatus, a high frequency level detecting unit detects a level of a high frequency component adjusted with a first adjusting unit, and a first control unit controls a first gain of the adjusting unit according to the level of the high frequency component thus detected. Further, a low frequency level detecting unit detects a level of a low frequency component adjusted with a second adjusting unit. A second control unit controls a second gain according to the level of the high frequency component and the level of the low frequency component thus adjusted, so that a difference between the level of the high frequency component adjusted with the first adjusting unit and the level of the low frequency component adjusted with the second adjusting unit becomes smaller than a specific level determined in advance.