A method for selecting a crop score threshold for enhancing tracking of objects in a scene captured in a video sequence is disclosed. A respective track is obtained for two different objects, each track comprising crops of object instances of the objects in in a video sequence, each crop having a crop score and a feature vector. Each track is split into respective more tracklets thereby forming four or more tracklets. For each candidate crop score threshold a respective difference between each tracklet and each other tracklet is determined based on differences between feature vectors of crops having a crop score above the candidate crop score threshold of each tracklet, and each other tracklet. A crop score threshold is selected from the set of crop score thresholds resulting in a maximum difference between the differences between tracklets of different tracks and the differences between tracklets of the same track.