Real-time facial recognition is augmented with a machine-learning process that samples pixels from images captured for the physical environmental background of a device, which captures an image of a user's face for facial authentication. The background pixel points that are present in a captured image of a user's face from a camera of the device are authenticated with the image of the user's face. The value of the background pixel points are compared against the expected values for the background pixel points provided by the on-going machine-learning process for the background.