A hearing device comprises a receiver, an input buffer and a sample processor, the receiver being adapted to receive samples of a digital audio signal and feed received samples as a digital input signal to the input buffer, the sample processor being adapted to process the buffered samples to provide samples of a digital output signal such that the digital output signal is a sample-rate converted representation of the digital input signal with a predetermined target sample rate. The hearing device further comprises a latency controller adapted to estimate the quality of reception of the digital audio signal and to control the processing of the buffered samples in dependence on the estimated quality of reception.