Techniques related to derivation of motion vectors of a first color component (e.g., chroma component) from motion vectors of a second color component (e.g., luma component) are described. A video coder (e.g., video encoder or video decoder), for a CU coded in affine mode with 4:4:4 color format, may determine a motion vector for each sub-block of the luma block, and determine a motion vector for each sub-block of the chroma block based only on the motion vector for each co-located (also called collocated) sub-block of the luma block. However, for another CU coded in affine mode but with a color format other than 4:4:4 (e.g., 4:2:2 or 4:2:0), the video coder may determine a motion vector for each sub-block of the chroma block based on an average of two or more motion vectors of sub-blocks of the luma block.