This disclosure describes a scaler architecture for image and/or video processing. One aspect relates to an apparatus comprising an image processing unit, a memory, and a coder. The memory is configured to store processed image data from the image processing unit. The coder is configured to retrieve the stored, processed image data from the memory. The coder comprises a scaler configured to upscale the retrieved image data from the memory. The coder is configured to encode the scaled image data.