An image processing apparatus includes a face detection unit configured to detect a face area of a person from an image, and a head detection area setting unit configured to set a head detection area based on the detected face area of the person. An edge detection unit is configured to detect an edge from the set head detection area, and to generate an edge image which is comprised of the detected edge. An edge deletion unit is configured to delete an edge existing between the face and hair from the edge image, and an ellipse detection unit is configured to detect a candidate ellipse corresponding to a head area from an edge image from which an edge is deleted by the edge deletion unit.