A method for image processing includes receiving a depth image of a scene containing a human subject and receiving a color image of the scene containing the human subject. A part of a body of the subject is identified in at least one of the images. A quality of both the depth image and the color image is evaluated, and responsively to the quality, one of the images is selected to be dominant in processing of the part of the body in the images. The identified part is localized in the dominant one of the images, while using supporting data from the other one of the images.