Systems and methods are disclosed to recognize human action from one or more video frames by performing 3D convolutions to capture motion information encoded in multiple adjacent frames and extracting features from spatial and temporal dimensions therefrom; generating multiple channels of information from the video frames, combining information from all channels to obtain a feature representation for a 3D CNN model; and applying the 3D CNN model to recognize human actions.