One embodiment provides an information processing apparatus effective to execute a parallel job in coordination with other information processing apparatuses. In an example, the information processing apparatus includes: a memory configured to store computer readable instructions; and a processor configured to execute the computer readable instructions sored in the memory, the computer readable instructions including: providing an instruction to issue barrier communication of error information; and propagating the error information to each of the other information processing apparatuses based on the instruction for the barrier communication.