Load balancing is described for a distributed computing system. For instance, resource usage information of devices in a distributed system is collected at a target device in the distributed system. Further a first work task for the target device is determined to be stopped based on the resource usage information, the target device having a first authority to execute the first work task. The first authority is then caused to be released. Beneficially, each node in the distributed system can individually balance different task loads and the use of resources by different operations of the task, thereby improving the performance of the distributed system.