Dual cameras that simultaneously capture RGB and IR images of a scene can be used to remove glare from the RGB image, transformed to a YUV image, by substituting a glare region in the luminance component of the YUV image with the pixel values in a corresponding region of the IR image. Further, color information in the glare region may be adjusted by averaging over or extrapolating from the color information in the surrounding region.