An object detection device comprises a specific object detector configured to detect a specific object in an image, a scene model creation unit configured to create a scene model characterizing a background of the specific object in the image, and a filtering unit configured to filter object detection results of the specific object detector using the scene model to determine the specific object, wherein the scene model creation unit comprises a collection unit configured to collect regions other than the specific object to be detected from the image as samples, a feature extraction unit configured to extract first negative feature vectors from the samples, a clustering unit configured to cluster first negative feature vectors into a plurality of feature groups, and a classifier creation unit configured to create first classifiers each for respective one of the feature groups and to create the scene model by combining the first classifiers.