Disclosed is a method that includes training, at a client, a part of a deep learning network up to a split layer of the client. Based on an output of the split layer, the method includes completing, at a server, training of the deep learning network by forward propagating the output received at a split layer of the server to a last layer of the server. The server calculates a weighted loss function for the client at the last layer and stores the calculated loss function. After each respective client of a plurality of clients has a respective loss function stored, the server averages the plurality of respective weighted client loss functions and back propagates gradients based on the average loss value from the last layer of the server to the split layer of the server and transmits just the server split layer gradients to the respective clients.