Systems and methods for controlling an unmanned aerial vehicle are disclosed. The system comprises an image sensor configured to generate output signals conveying visual information, the visual information including one or more images of a user, and one or more physical processors. The one or more physical processors are configured by computer-readable instructions to recognize one or more gestures from the user based on the visual information, interpret the one or more gestures from the user as flight control information, and provide flight control for the unmanned aerial vehicle based on the flight control information.