An apparatus for controlling display of a stereoscopic image, includes: a display control portion which cuts out a stereoscopic image in a to-be-displayed area which is an area corresponding to a part of a stereoscopic image consisting of a plurality of images with a parallax, and displays the cut-out stereoscopic image on a display portion; a parallax change amount calculation portion which calculates a change amount between a parallax amount in a stereoscopic image within the to-be-displayed area before the to-be-displayed area is moved and a parallax amount in a stereoscopic image within the to-be-displayed area after the to-be-displayed area is moved by a first moving distance when an instruction to move the to-be-displayed area is issued; and a to-be-displayed area movement control portion which moves the to-be-displayed area by a moving distance in accordance with the change amount calculated by the parallax change amount calculation portion.