An apparatus and method for processing a holographic image. The apparatus calculates a first calculation result with respect to an image for the left eye and a first calculation result with respect to an image for the right eye and stores the results at different memory addresses of a storage. Thereafter, the apparatus calculates values of a waveform of light to be modulated by a spatial light modulator by performing a second calculation that uses all of the first calculation results stored in the storage. An image window of the image for the left eye and an image window of the image for the right eye are spatially separated from each other by the apparatus in a viewing window of a hologram image reproduced via the spatial light modulator.