This document describes a system for animating a two-dimensional object in real-time with gestural data collected by a gestural sensor of the system. A gestural sensor collects gestural data. An image extraction engine extracts image data that renders a two-dimensional object on a display device of the system. An overlay engine generates a mesh overlay for the two-dimensional object. Based on a detected gesture represented in the gestural data collected, an animation engine modifies portions of the two-dimensional object from a first image frame to a second image frame based on the collected gestural data. An image frame generator generates the first and second image frames that animate the two-dimensional object in accordance with the gestural data collected.