Various systems and methods for providing a vision-based robot control system are provided herein. A vision-based robot control system comprises a camera system interface to receive image data from a camera system; a trigger detection unit to determine a triggering action from the image data; and a transceiver to initiate a robot operation associated with the triggering action.