A system and a method perform matrix factorization. According to the system and the method, at least one matrix is received. The at least one matrix is to be factorized into a plurality of lower-dimension matrices defining a latent feature model. After receipt of the at least one matrix, the latent feature model is updated to approximate the at least one matrix. The latent feature model includes a plurality of latent features. Further, the update performed by cycling through the plurality of latent features at least once and alternatingly updating the plurality of lower-dimension matrices during each cycle.