The application relates to a hearing device comprising a) a forward path between an input transducer for converting an input sound to an electric input signal and an output transducer for converting an electric output signal to an output sound, the forward path comprising a signal processing unit for applying a level and/or frequency dependent gain to the electric input signal or a signal originating therefrom and for providing a processed signal, and feeding the processed signal or a signal originating therefrom to the output transducer, an acoustic feedback path being defined from said output transducer to said input transducer; b) a configurable anti-feedback system comprising a feedback estimation unit for providing an estimate of said acoustic feedback path; c) a number of detectors, each providing a detector signal for characterizing a signal of the forward path. The object of the present application is to save power in a hearing device. The problem is solved in that the hearing device further comprises an activation control unit configured to control the anti-feedback system based on said detector signals, and to bring the anti-feedback system into one of at least two predefined modes based on said detector signals, said at least two predefined modes comprising an ON-mode and an OFF-mode. The invention may e.g. be used in hearing aids, headsets, ear phones, active ear protection systems, or similar portable devices, where a need for feedback cancellation and low power consumption is important.