Asymmetries are detected in one or more images by partitioning each image to create a set of patches. Salient patches are identified, and an independent displacement for each patch is identified. The techniques used to identify the salient patches and the displacement for each patch are combined in a function to generate a score for each patch. The scores can be used to identify possible asymmetries.