A parallel processing system includes a plurality of computers accessibly connected through a network, and distributedly executing a plurality of processes. Each of the plurality of computers is composed of an operation processing unit configured to execute an allocated process, a local memory group having a first area and a second area and an I/O control circuit. The operation processing unit executes the allocated process by using the first area as an access destination in a first period and, and executes the allocated process by using the second area as the access destination in a second period subsequent to the first period. The I/O control circuit is composed of an updating section configured to update data stored in the local memory group to the latest data by carrying out communication among the computers. The updating section updates the data stored in the first area in the second period.