A method of handling exceptions in a data parallel system includes forwarding exceptions thrown by concurrent worker tasks to a coordination task. The thrown exceptions are aggregated into an aggregation exception structure. It is determined whether the aggregation exception structure will be handled by an exception handler. The concurrent worker tasks are unwound when it is determined that the aggregation exception structure will be handled.