This is a video image monitoring system which can effectively detect a mobile object appearing in a captured video image even if a background image and other camera condition change continuously. The video image monitoring system comprises: a video-image-capturing section 100 for putting out image data based on a video image signal obtained by using a camera 10; a mobile-object-candidate-area-detecting section 101 for extracting a candidate area of a mobile object from the image data; and a mobile-object-detecting section 102 for determining whether the candidate area is the mobile object. The mobile-object-candidate-area-detecting section 101 quantizes a brightness gradient direction of the image data, and calculates a spatio-temporal histogram which represents the frequency of a direction code appearing in a predetermined spatio-temporal space. After that, the mobile-object-candidate-area-detecting section 101 calculates a statistical spatio-temporal space evaluation value of the spatio-temporal histogram. The mobile-object-detecting section 102 uses the spatio-temporal space richness to determine whether the candidate area is the mobile object.