Techniques for video encoding and decoding are described. An example method of video processing is disclosed. The method includes classifying motion candidates in a candidate list into a plurality of categories of motion candidates, each category being assigned a corresponding pruning rule; performing pruning or skipping the pruning of the motion candidates in the candidate list based on the corresponding pruning rule, the pruning being applied to determine whether to insert motion candidates from the candidate list into a final candidate list for a current block; and performing a video processing on the current block based on the final candidate list.