A position measuring system includes: an image capturing unit that captures reference points provided on an object, the reference points composed of at least four first reference points provided respectively at vertices of a polygon or at vertices and a barycenter of a polygon and at least one second reference point provided so as to have a specific positional relationship with respect to the first reference points; an identification unit that identifies images of the first reference points and the second reference point captured by the image capturing unit, on the basis of positional relationships between the images of the first reference points and the second reference point; and a calculation unit that calculates a three-dimensional position and three-axial angles of the object on the basis of positional relationships of the images of the first reference points identified by the identification unit.