Systems and methods for handling color vision deficiencies in virtual, augmented, and mixed reality (xR) applications are described. In some embodiments, a hardware memory device may have program instructions stored thereon that, upon execution by a hardware processor of an Information Handling System (IHS), cause the IHS to: apply a first set of RGB adjustments to a foreground portion of an image to be displayed by Head-Mounted Device (HMD) worn by a user; apply a second set of RGB adjustments to a background portion of the image, where the first and second set of RGB adjustments produce a visual indication of the presence of a color, in the image, that the user is incapable of naturally identifying; and display the visual indication.