Disclosed herein is robust detection of facial expressions that accounts for interferences on the face. In one embodiment, a system includes a head-mounted device that comprises (i) light sources configured to emit light towards a first region on a user's face, and (ii) discrete photosensors, spread over more than 2 cm, which take measurements of reflections of the light from the first region. Additionally, the system includes a head-mounted camera that captures images of a second region on the face. A computer calculates, based on the images, an extent of presence of hair over a portion of the first region. Optionally, the computer calculates extents of other interferences such as perspiration, applied makeup, and skin infection. The computer then detects facial expressions of the user based on the measurements of the reflections and the calculated extent of the hair and/or other interferences.