Techniques to improve the quality of captured images by reducing the effects of undesired objects (e.g., screen glare) are disclosed. The techniques may involve the use of face detection to localize the likely position of screen glare within the captured images (e.g., on a user's eyeglasses), as well as an awareness of the content that is being displayed on a display screen (or other light-projecting element projecting light into the scene) at the moment of capture of the respective image. The techniques may then model the position, size, and/or distortion of the screen contents (or other projected light) reflected by the user's eyeglasses (or other reflective surface in the captured scene environment). Once the appearance of the undesired screen glare has been modeled in the captured image, the techniques may perform an image modification operation to remove or reduce the undesired glare from the originally-acquired image in an efficient manner.