Specified data is distributed in a High-Performance Computing cluster comprising a Management node and M computation nodes where M is an integer greater than one, by the method comprising: dividing the M computation nodes into m layers where m is an integer greater than one; dividing the specified data into k shares where k is an integer greater than one; distributing, by the Management node, the k shares of data to a first layer of computation nodes as sub-nodes thereof, each node of the first layer obtaining at least one share of data therein; distributing, by each of the computation nodes, the share(s) of data distributed by a parent node thereof to sub-nodes thereof; and requesting, by each of the computation nodes, the remaining specified data to other computation nodes, to thereby obtain all the specified data.