A computer-implemented technique is described herein for invalidating pixels in a time-of-flight depth-sensing device based on active brightness (AB) measurements. In one implementation, the technique involves, for each sensing element of a sensor: generating frequency-specific sensor readings in response to receiving instances of radiation having plural frequencies (e.g., frequencies f1, f2, and f3); generating a set of active brightness measurements (ABf1, ABf2, and ABf3) associated with the respective frequencies; generating a variation measure that reflects an extent of variation within the set of active brightness measurements; and invalidating a pixel associated with the particular sensing element if the variation measure satisfies a prescribed invalidation condition.