Technologies are presented that allow efficient pixel-based image and/or video warping and scaling. An image processing system may include a memory and an accelerator unit communicatively coupled with the memory. The accelerator unit may, based on configuration settings, receive, from a memory, at least a portion of an input image as an array of neighboring four-cornered shapes; and process each shape by: determining locations of an array of output pixels delineated by four corner locations of the shape via linearization; interpolating a value of each pixel of the array of output pixels; and storing the interpolated pixel values in the memory. For warping, the array of neighboring four-cornered shapes may include an array of neighboring distorted tetragons that approximate distortion of the input image, and the interpolated pixel values may represent a warped output image. For scaling, the array of neighboring four-cornered shapes may include an array of neighboring rectangles.