A hearing device, e.g. a hearing aid, comprises a forward path comprising an input transducer providing an electric input signal, a combination unit, a signal processing unit configured to apply a forward gain to signal of the forward path and to provide a processed electric output signal, a frequency shifting unit for de-correlating the processed electric output signal and the electric input signal, and an output transducer. The hearing device further comprises an adaptive filter for providing an estimate of an external feedback path, and located in the forward path. The feedback estimation unit provides a resulting feedback estimate signal, which is combined with the electric input signal in the combination unit to provide a resulting feedback corrected signal, and a correction unit for influencing said estimate of the feedback path by diminishing a residual bias, being a result of the frequency shift, in said resulting feedback estimate signal.