Techniques are provided to improve interaction between a user and a projection system. In some embodiments, an image of a user in front of a display screen is captured. An inference can then be made as to whether a user is touching a display screen based on an analysis of shadows and/or variation of brightness (i.e., intensities) across pixels in the image. For example, it may be inferred that the object is: (1) approaching the screen when a region surrounding a top of the object is characterized by a relatively small brightness variation; (2) hovering near the screen when the brightness variation is large and the region includes a dark extremum (caused by a shadow); and (3) touching the screen when the brightness variation is large and the region includes a light extremum.