An encoder for encoding an audio signal has a predictor, a factorizer, a transformer and a quantize and encode stage. The predictor is configured to analyze the audio signal to obtain prediction coefficients describing a spectral analog of the audio signal or a fundamental frequency of the audio signal and subject the audio signal to an analysis filter function dependent on the prediction coefficients to output a residual signal of the audio signal. The factorizer is configured to apply a matrix factorization onto an audiocorrelation or covariance matrix of synthesis filter function defined by the prediction coefficients to obtain factorized matrices. The transformer is configured to transform the residual signal based on the factorized matrices to obtain a transformed residual signal. The quantize and decode stage is configured to quantize the transformed residual signal to obtain a quantized transformed residual signal or an encoded quantized transformed residual signal.