We describe techniques for restoring an audio signal. In embodiments these employ masked positive semi-definite tensor factorization to process the signal in the time-frequency domain. Broadly speaking the methods estimate latent variables which factorize a tensor representation of the (unknown) variance/covariance of an input audio signal, using a mask so that the audio signal is separated into desired and undesired audio source components. In embodiments a masked positive semi-definite tensor factorization of ψftk=MftkUfkVtk is performed, where M defines the mask and U, V the latent variables. A restored audio signal is then constructed by modifying the input signal to better match the variance/covariance of the desired components.