An artificial neural network for learning to track a target across a sequence of frames includes a representation network configured to extract a target region representation from a first frame and a search region representation from a subsequent frame. The artificial neural network also includes a cross-correlation layer configured to convolve the extracted target region representation with the extracted search region representation to determine a cross-correlation map. The artificial neural network further includes a loss layer configured to compare the cross-correlation map with a ground truth cross-correlation map to determine a loss value and to back propagate the loss value into the artificial neural network to update filter weights of the artificial neural network.