A method for reducing memory utilization in a digital video codec. The method generally includes the steps of (A) generating a second reference picture by downsampling a first reference picture using a pattern, wherein the pattern (i) comprises a two-dimensional grid and (ii) is unachievable by performing a vertical downsampling and separately performing a horizontal downsampling, (B) generating a third reference picture by upsampling the second reference picture and (C) processing an image in a video signal using the third reference picture.