A method of processing an image to form an image pyramid having multiple image levels includes receiving a base level image comprising pixel values at pixel locations arranged in rows and columns; determining sample locations for a next level image in the pyramid such that the sample locations are arranged in a regular pattern and the sample locations exceed the range of the pixel locations of the base level image; determining the pixel values of the next level image by interpolating the pixel values of the base level image using an interpolation filter at the sample locations; and treating the next level image as the base level image and repeating steps of determining sample locations and pixel values until a predetermined number of pyramid image levels are generated, or until a predetermined condition is met.