Methods and systems are disclosed to aid in the detection of cancer or lesion in a mammogram images. Two mammogram images are input into an application that aids in determining the probability of a cancer or lesion being present in one or both of the images. The images are divided into different nodes and labels are applied to the nodes. The first node is compared to different variants of corresponding nodes on the second image as well as neighboring nodes on the first image. Based upon the comparisons, a unary and binary potential is calculated for the label that is applied to the node. The process is repeated for every possible label and for every node. Once the unary and binary potentials have been calculated, the potentials are input into a Conditional Random Field model to determine the probability of cancer for each node of the images.