A method for detecting presence of a hand gesture in video frames includes receiving video frames having an original resolution, downscaling the received video frames into video frames having a lower resolution, and detecting a motion corresponding to the predefined hand gesture in the downscaled video frames based on temporal motion information in the downscaled video frames. The method also includes detecting a hand shape corresponding to the predefined hand gesture in a candidate search window within one of the downscaled video frames using a binary classifier. The candidate search window corresponds to a motion region containing the detected motion. The method further includes determining whether the received video frames contain the predefined hand gesture based on the hand shape detection.