Pixels of a binary image obtained by binarizing an image are scanned in a predetermined direction, labels are assigned to the pixels according to binarization information about the respective pixels, information about the assigned labels is stored sequentially for each of a plurality of lines along the predetermined direction, information about coordinate values in the binary image of pixels assigned the same label is stored, a determination is made as to whether or not, in a current line among the plurality of lines, there is a pixel assigned the same label as a label assigned to a pixel contained in a line which was scanned immediately before the current line, when a determination is made that there is no pixel assigned the same label, a feature point in a connected component formed by connecting together pixels specified by the coordinate values is calculated based on the stored information about the coordinate values, a feature vector representing a feature of the image is calculated based on the calculated feature point, and a similarity to reference image is determined based on the calculated feature vector.