An information handling system manages operation of an infrared time of flight sensor to provide accurate and timely user presence and absence detection through adjustments of the time of flight sensor detection sensitivity based upon ambient light brightness and color temperature sensed by an ambient light sensor. An integrated sensor hub in a central processing unit disables infrared illumination by the time of flight sensor, senses ambient light conditions with the ambient light sensor, looks up sensitivity settings from a lookup table that associates infrared time of flight sensor sensitivity and ambient light conditions, applies the associated sensitivity at the infrared time of flight sensor and then re-enables infrared illumination to detect end user presence and absence with the infrared time of flight sensor.