Embodiments of the present disclosure relate to weight matrix prediction. In an embodiment, a computer-implemented method is disclosed. The method comprises sending a candidate weight matrix of a neural network to one of a plurality of computing nodes comprised in a computing system to perform a testing iteration. The method further comprises receiving a testing loss value from the one of the plurality of computing nodes based on the testing iteration. The method further comprises evaluating whether the testing loss value is applicable. The method further comprises determining that the candidate weight matrix is available to be employed in a new formal iteration in response to the testing loss value being applicable. In other embodiments, a system and a computer program product are disclosed.