The present disclosure describes systems and techniques relating to identifying and tracking objects in images, such as visual tracking in video images in unconstrained environments. According to an aspect, a system includes one or more processors, and computer-readable media configured and arranged to cause the one or more processors to: identify an object in a first image of a sequence of images, identifying one or more regions similar to the object in the first image of the sequence of images, identifying one or more features around the object in the first image of the sequence of images, preventing drift in detection of the object in a second image of the sequence of images based on the one or more regions similar to the object, and verifying the object in the second image of the sequence of images based on the one or more features.