Techniques for controlling an augmented reality object are described in various implementations. In one example implementation, a method may include receiving an initialization image captured by an image capture device, the initialization image depicting a background and being free of foreground objects positioned between the background and the image capture device. The method may also include receiving a plurality of subsequent images captured by the image capture device over a period of time, the plurality of subsequent images depicting the background and a foreground object, the foreground object being positioned between the background and the image capture device. The method may also include comparing the initialization image to the plurality of subsequent images to determine positioning of the foreground object over the period of time. The method may also include controlling an augmented reality object based on the positioning of the foreground object over the period of time.