An activity recognition method, for recognizing continuous activities of several moving objects in the foreground of a video, includes: capturing and processing a training video to get a contour of a moving object; extracting a minimum bounding box of the contour in order to get parameters then transfer to feature vectors; constructing a decision tree model based on support vector machines (SVMs), for classifying the activities of the moving object according to the parameter and the feature vector of the training video; capturing and processing a testing video to get other parameters and using several formulas to generate feature vectors, and executing an algorithm for recognizing the activities of several moving objects in the foreground of the testing video. Said feature vectors are transformed from the parameters that in the testing and training videos. Via above descriptions, the method can recognize activities of foreground objects in the testing video.