An impulse noise remover includes a delay chain, a plurality of detection modules coupled to the delay chain, a logic unit coupled to the detection modules, and a multiplexer coupled to the delay chain and the logic unit. The delay chain delays the input signal to generate a plurality of delayed signals. The delayed signals and the input signal constitute a reference signal set. Each of the detection modules generates an indication signal according to a threshold value and a signal subset of the reference signal set. The logic unit generates a control signal according to a plurality of indication signals generated by the detection modules. The multiplexer selectively outputs a padding value or one of the delayed signals as the output signal according to the control signal.