A low latency relinearization process can be performed in an FPGA cluster for accelerating homomorphic encryption. The low-latency process performs an early calculation of matrix rows to make the summation result available earlier in the relinearization to reduce waiting of subsequent operations.