Techniques are disclosed for determining a user's motion in relation to displayed images. According to one general aspect, a first captured image is accessed. The first captured image includes (1) a first displayed image produced at a first point in time, and (2) a user. A second captured image is accessed. The second captured image includes (1) a second displayed image produced at a second point in time, and (2) the user. First information indicating motion associated with one or more objects in the first and second displayed images is accessed. Second information indicating both motion of the user and the motion associated with the one or more objects in the first and second displayed images is determined.