Embodiments of the present disclosure relates to constraints for usage of updated motion information. A method for video processing is provided, including: determining original motion information associated with a current block; generating updated motion information based on specific prediction mode; and performing, based on the updated motion information, a conversion between the current block and a bitstream representation of a video data including the current block, wherein the specific prediction mode includes one or more of bi-directional optical flow (BIO) refinement, a decoder-side motion vector refinement (DMVR), frame-rate up conversion (FRUC) techniques or a template matching technique.