A system and method determining a camera pose. The method comprises receiving a first image and a second image, the first and second images being associated with a camera pose and a height map for pixels in each corresponding image, and determining a mapping between the first image and the second image using the corresponding height maps, the camera pose and a mapping of the second image to an orthographic view. The method further comprises determining alignment data between the first image transformed using the determined mapping and the second image and determining a refined camera pose based on the determined alignment data and alignment data associated with at least one other camera pose.