The present invention discloses a stereoscopic image display system and a method of controlling the same. An eye tracking module locates current 3D spatial positions of the viewer's eyes, and generates the information of both left and right eyes' current 3D spatial positions. A control module controls a display device that can alter the direction of the light outputted, and outputs images on the display device in time multiplex mode. The light containing the left eye image is outputted to the position of left eye instead of right eye at one time point, and the light containing the right eye image is outputted to the position of right eye instead of left eye at another time point, so that a stereoscopic image is perceived according to the parallax theory. The present invention enlarges the visual range of stereoscopic image and achieves a better stereoscopic image visual experience for viewers.