A method for operation a terminal device with gesture and a device are provided. The method includes obtaining a gesture video segment comprising a preset number of frames of images, obtaining gesture track information of a user according to a location of a finger of the user in each frame of image of the gesture video segment, searching in a preset corresponding relationship between the gesture track information and an operation according to the gesture track information, obtaining the operation corresponding to the gesture track information of the user and performing the operation. With the technical scheme of the present disclosure, the gesture track information of the user is obtained via analyzing each frame of image in the obtained gesture video segment. The gesture track information of the user in a dimensional plane is concerned and the corresponding operation thereof is obtained via the gesture track information. This technical scheme may enrich the methods for triggering the operation, operate the terminal device without directly touching the terminal device and reduce the dependence on the peripheral devices, such as the mouse and the keyboard.