The disclosure relates to a method and apparatus for training a learning machine, wherein the apparatus includes: a broadcasting module for broadcasting an initial global model for a training cycle to a plurality of worker nodes; a receiving module for receiving a plurality of updated local models from the plurality of worker nodes, wherein each updated local model is generated by one of the plurality of worker nodes independently based on a data split assigned to the worker node and the initial global model for the training cycle; an aggregating module for aggregating the plurality of updated local models to obtain an aggregated model; and a generating module for generating an updated global model for the training cycle based at least on the aggregated model and historical information which is obtained from a preceding training cycle.