A method for tracking objects in a scene being viewed by a sequence of digital images comprises: separating a region of interest in the digital images into a plurality of spatially smaller, overlapping modules, and defining a hidden Markov model for each module. The method further comprises observing detections of positions of possible objects at different times, considering a set of states at a point of time (t1) such that the set of states comprises a state having a global optimum probability, and backtracking through the sequence of digital images for each hidden Markov model to shrink the set of states that are possible for earlier parts of the sequence of digital images, such that a single optimal state is found for an earlier point of time (t2