Disclosed herein are a method of transferring data in a parallel system including a main device and at least one accelerator, and a parallel system for performing the method. The method of transferring data in a heterogeneous system including a main device and at least one accelerator includes: turning off a write permission for a first main memory area corresponding to a first accelerator memory area where input data for a computation task is stored; performing the computation task by using the at least one accelerator; and turning off a read permission for a second main memory area corresponding to a second accelerator memory area where output data for the computation task is stored, in the state in which data of the second accelerator memory area has not been transferred to the second main memory area.