Systems, methods, and computer program products for facilitating a serial debugging experience for programs running in a distributed, non-deterministic, out-of-order (i.e., parallel) fashion are disclosed. Such systems, methods, and computer program products provide a modified combination of breakpoints, stepping, and call stacks that facilitate the ability to improve the debugging experience for out-of-order program execution. In an embodiment, a task scheduler provides a distributed task graph. Then, the user may hit breakpoints sequentially within the graph's execution and use deterministic stepping (i.e., step-in, step-out, and step-over functions) to create a logical call stack from a distributed task graph. Such embodiments allow the program to execute in a distributed and parallel manner, while presenting the user a sequential execution experience in the debugger.