An infrared (IR) imaging module may capture a background image in response to receiving IR radiation from a background of a scene and determine background calibration terms using the background image. The determined background calibration terms may be scale factors and/or offsets that equalize the pixel values of the background image to a baseline, value. IR imaging device may use the background calibration terms to capture images that have the baseline value for pixels corresponding to IR radiation received from the background and higher values (or lower values) for pixels corresponding to IR radiation received from a foreground. Such images may be used to count people and generate a heat map. The background calibration terms may be updated periodically, with the update period being increased at least for some pixels or a pixel area when a person is detected.