In general, a system is described that includes a set of one or more cameras and a computing device. The computing device receives a plurality of images of a three-dimensional environment captured by the one or more cameras, and a respective camera that captures a respective image is distinctly positioned at a respective particular location and in a respective particular direction. The computing device generates a plurality of image sets that each include at least three images. For each image set, the computing device calculates a plurality of predicted pairwise directions. The computing device compares a first sum of model pairwise directions with a second sum of the plurality of predicted pairwise directions and generates an inconsistency score for the respective image set. The computing device then reconstructs a digital representation of the three-dimensional environment depicted in the images.