An image processing apparatus includes a moving image input unit configured to input a moving image, an object likelihood information storage unit configured to store object likelihood information in association with a corresponding position in an image for each object size in each frame included in the moving image, a determination unit configured to determine a pattern clipping position where a pattern is clipped out based on the object likelihood information stored in the object likelihood information storage unit, and an object detection unit configured to detect an object in an image based on the object likelihood information of the pattern clipped out at the pattern clipping position determined by the determination unit.