A motion detection method determines bit-rate changes of input pixels of a video frame by a grey relational analysis technique to establish a multi-quality background model, detects moving objects by two-stage block-based and pixel-based detection procedures to generate a binary motion mask, detects luminance changes of the video frame by entropy calculation to timely update the background model, provides a setting interface for a user to set a detection sensitivity, and examines false detections of the binary motion mask. Therefore, it can correctly interpret moving objects in VBR video streams, implement more accurate and complete motion detection, eliminate the influence of luminance changes, increase the detection accuracy, and decrease false detections.