A hearing device, e.g. a hearing aid or a headset, configured to be worn by a comprises an input unit for providing at least one electric input signal in a time-frequency representation; and a signal processor comprising a target signal estimator for providing an estimate of the target signal; a noise estimator for providing an estimate of the noise; and a gain estimator for providing respective gain values in dependence of said target signal estimate and said noise estimate. The gain estimator comprises a trained neural network, wherein the outputs of the neural network comprise real or complex valued gains, or separate real valued gains and real valued phases. The signal processor is configured—at a given time instance t—to calculate changes Δx(i,t)=x(i,t)−{circumflex over (x)}(i,t−1), and Δh(j,t−1)=h(j,t−1)−ĥ(j,t−2) to an input vector x(t) and to the hidden state vector h(t−1), respectively, from one time instance, t−1, to the next, t, and where {circumflex over (x)}(i,t−1) and ĥ(j,t−2) are estimated values of x(i,t−1) and h(j,t−2), respectively, where indices i, j refers to the ith input neuron and the jth neuron of the hidden state, respectively, where 1≤i≤Nch,x and 1≤j≤Nch,oh, wherein Nch,x and Nch,oh is the number of processing channels of the input vector x and the hidden state vector h, respectively, and wherein the signal processor is further configured to provide that the number of updated channels among said Nch,x and said Nch,oh processing channels of the modified gated recurrent unit for said input vector x(t) and said hidden state vector h(t−1), respectively, at said given time instance t is limited to a number of peak values Np,x, and Np,oh, respectively, where Np,x is smaller than Nch,x, and Np,oh, is smaller than Nch,oh.