Camera pose optimization, which includes determining the position and orientation of a camera in three-dimensional space at different times, is improved by detecting a higher-confidence reference object in the photographs captured by the camera and using the object to increase consistency and accuracy of pose data. Higher-confidence reference objects include objects that are stationary, fixed, easily recognized, and relatively large. In one embodiment, street level photographs of a geographic area are collected by a vehicle with a camera. The captured images are geo-coded using GPS data, which may be inaccurate. The vehicle drives in a loop and captures the same reference object multiple times from the substantially same position. The trajectory of the vehicle is then closed by aligning the points of multiple images where the trajectory crosses itself. This creates an additional constraint on the pose data, which in turn improves the data's consistency and accuracy.