An image search apparatus extracts a symmetric partial image based on acquired pairs of feature points, and divides the partial image at a symmetry axis of the partial image to decide two partial areas, as a first area and a second area based on the tendency of image features in the partial image. In determining the similarity between the query image and a comparison destination image, a coordinate transformation coefficient for a coordinate transformation process is set based on pairs of feature points extracted from the partial area decided as the first area of the two images, the coordinate transformation process using the coordinate transformation coefficient is applied to the pairs of feature points extracted from areas including areas other than the first area of the two images, and coordinates of the pairs of feature points after the coordinate transformation process are compared.