An apparatus and method for LU decomposition of an input matrix, the input matrix comprising a multitude of elements forming a plurality of rows and columns. In an embodiment, the apparatus comprises a memory including a plurality of memory caches, and a processor unit operatively connected to the memory to transmit data to and receive data from the memory caches. The processing unit comprises a hardware circuit for processing the input matrix column-by-column to decompose the input matrix into a lower triangular matrix L and an upper triangular matrix U, including performing Gaussian eliminations on the columns of the matrix, with partial pivoting of the matrix, and choosing one of the elements of each of the columns as a pivot element for said each column while said each column is being processed.