A video processing method includes performing or skipping following processing based on whether a size of a coding unit is not smaller or is smaller than 8×8. The processing includes dividing the coding unit into sub-blocks each having a fixed size of 8×8, scanning a left neighboring block and determining a reference motion vector of the coding unit, determining a related reference block of a sub-block in the co-located reference image according to the reference motion vector, determining a scaling factor of a motion vector of the related reference block, scaling the motion vector of the related reference block using the scaling factor, determining motion information of the sub-block according to the scaled motion vector, and performing prediction for the coding unit according to the motion information.