Computer vision applications often require each pixel within an image to be assigned one of a set of labels. A method of improving the labels assigned to pixels is described which uses the quadratic pseudoboolean optimization (QPBO) algorithm. Starting with a partially labeled solution, an unlabeled pixel is assigned a value from a fully labeled reference solution and the energy of the partially labeled solution plus this additional pixel is calculated. The calculated energy is then used to generate a revised partially labeled solution using QPBO.