Systems and methods for gesture recognition based on depth information from a camera include, at an electronic device having a camera system, capturing a video frame and depth information associated with the video frame, identifying a foreground portion of the video frame based on the depth information, and determining whether the foreground portion matches a predefined gesture in a database. In accordance with a determination that the foreground portion matches a predefined gesture in the database, the device determines whether one or more subsequent video frames matches the one or more predefined gestures to produce a recognized gesture.