A method and apparatus adapts motion vector prediction for suitability to omnidirectional video. One embodiment improves handling of temporal motion vector predictors or rescaled motion vector predictors. Another embodiment is suited to spatial motion vector predictors, and another to a combination of either temporal or spatial motion vector predictors. The method analyzes a scale factor derived from, at least one of, the time index of the predictor, the time index of the reference image's predictor, the time index of a reference image's current block, and the time index of the current block. If, for example, the scale factor is greater than one, motion vector transformation is performed before motion vector rescaling. If, however, the scale factor is less than or equal to one, the motion vector rescaling is performed before motion vector transformation.