Efficient search window storage schemes for motion estimation in video signal processing are disclosed. According to one embodiment, motion estimation is carried out as follows: allocating a ring buffer to store at least critical number of macro-blocks of luminance data from a reference frame on the motion estimator, establishing a rule to relate each storage unit of the ring buffer with each of the macro-blocks of the reference frame, initializing the ring buffer by reading in one less than the critical number of macro-blocks from the reference frame, when the macro-block is located next to a border of the reference frame, generating one or more added border macro-blocks and storing into the corresponding position of the ring buffer in accordance with the rule, constructing the search window from the macro-blocks stored in the ring buffer, and conducting motion estimation of the current macro-block of the current frame with the search window.