Aspects of the disclosure provide a video coding method for processing a current prediction unit (PU) with a sub-PU temporal motion vector prediction (TMVP) mode. The method can include receiving the current PU including sub-PUs, determining an initial motion vector that is a motion vector of a spatial neighboring block of the current PU, performing a searching process to search for a main collocated picture in a sequence of reference pictures of the current PU based on the initial motion vector, and obtaining collocated motion information in the main collocated picture for the sub-PUs of the current PU. The searching process can include turning on motion vector scaling operation for searching a subset of the sequence of reference pictures, and turning off the motion vector scaling operation for searching the other reference pictures in the sequence of reference pictures.