A hearing system comprises a) a multitude of M of microphones providing M corresponding electric input signals xm(n), m=1, . . . , M, and n representing time, b) a processor connected to said multitude of microphones and providing a processed signal in dependence thereof, c) an output unit for providing an output signal in dependence of said processed signal, and d) a database (Θ) comprising a dictionary (Δpd) of previously determined acoustic transfer function vectors (ATFpd). The processor is configured A) to determine a constrained estimate of a current acoustic transfer function vector (ATFpd,cur) in dependence of said M electric input signals and said dictionary (Δpd), B) to determine an unconstrained estimate of a current acoustic transfer function vector (ATFuc,cur) in dependence of said M electric input signals, and C) to determine a resulting acoustic transfer function vector (ATF*) for a user of the hearing system in dependence thereof and of a confidence measure related to said electric input signals. A method of operating a hearing device is also disclosed. Thereby an improved noise reduction system for a hearing aid or headset may be provided.