A gesture control system capable of interacting with three-dimensional (3D) images comprises a 3D image display device, a gesture recognition device, and a data processing unit. The 3D image display device displays a 3D image. The gesture recognition device captures a hand(s) and finger(s) image of a user and then calculates a hand(s) and finger(s) coordinate. The hand(s) and finger(s) coordinate is transmitted to the data processing unit and is matched with an image 3-dimensional spatial coordinate for being output via the 3D image display device in order to achieve advantages of interacting with 3D images.