Embodiments of the invention provide a method and a system for tracking an object from a training image to a target image. The training image and the target image are elements of a sequence of images. The object in the training image is represented by an object state. First, a set of particles is acquired, wherein each particle in the set of particles is associated with a weight, such that the particle represents the object state with a probability equal to the weight. Next, a regression function is applied to each particle in the set of particles based on a target image to determine a set of moved particles and the object state is updated according to the set of moved particles, such that the object state represents the object in the target image.