A position/orientation measurement apparatus holds a three-dimensional shape model of a object, acquires approximate value indicating a position and an orientation of the object, acquires a two-dimensional image of the object, projects a geometric feature of the three-dimensional shape model on the two-dimensional image based on the approximate value, calculates the direction of the geometric feature of the three-dimensional shape model projected on the two-dimensional image, detects an image feature based on the two-dimensional image, calculates the direction of the image feature, associates the image feature and the geometric feature by comparing the direction of the image feature calculated based on the two-dimensional image and the direction of the geometric feature calculated based on the three-dimensional shape model, and calculates the position and orientation of the object by correcting the approximate value based on the distance between the geometric feature and the image feature associated therewith.