The present disclosure provides a method, a device and a readable storage medium for processing a control instruction based on gesture recognition, the method including: obtaining a trigger-gesture-recognition instruction; capturing a set of to-be-recognized images corresponding to a to-be-recognized user according to the trigger-gesture-recognition instruction, where the set of to-be-recognized images includes to-be-recognized images of multiple frames; determining location information of a hand of the to-be-recognized user respectively in each frame of the to-be-recognized images via a preset body recognizing model; and according to the location information, obtaining gesture information corresponding to the hand via a preset gesture recognizing model; and according to the gesture information corresponding to the hand, obtaining a control instruction corresponding to the gesture information. Thus, the amount of computation for gesture recognition is reduced, and efficiency of gesture recognition is improved, enabling a remote gesture control of an intelligent terminal.