A noise removing apparatus and method usable in a portable recorder to remove noise and enhance a voice signal when the voice signal with noise is input to the portable recorder. The noise removing method includes dividing an input signal into a high frequency component signal and a low frequency component signal, adjusting a magnitude of the high frequency component signal by comparing the magnitude of the divided high frequency component signal and a magnitude of the low frequency component signal, enhancing a clearness of the low frequency component signal by filtering the divided low frequency component with an adaptive filter, and adding the magnitude-adjusted high frequency component signal and the clearness-enhanced low frequency component signal.