In response to receiving a checkpoint request from a first task for a process executing in parallel with at least a second task for the process, checkpoint data from the first task is stored to establish a restart point for the first task, wherein the checkpoint data records a current state of the first task, and the checkpoint data from the first task is merged with previously received checkpoint data for the at least second task to create a checkpoint file. In response to restarting the process, the first task and the at least second task are restarted using the checkpoint data in the checkpoint file.