Structural content filtration to reduce the number of hypotheses for the location of an active object in a recorded image generated by a graphical user interface (GUI) of an application program may be accomplished by transforming the recorded image and a corresponding playback image, determining a sub-image from the transformed recorded image which corresponds to an object to be searched for in the transformed playback image, determining a set of points on the transformed playback image which have appropriate values for matching the sub-image, and filtering hypotheses on the playback image which are more than a selected distance from any one of the subset of points.