Embodiments of the present invention are directed to optimizing image cropping. In accordance with some embodiments of the present invention, an image and an indication of an area of interest within the image are obtained. Thereafter, an amount to scale the image is determined based on a size of a container into which the image is to be placed for display. The amount to scale the image is greater for containers of a smaller size to focus on the area of interest within the image than the amount to scale the image for containers of a larger size. The image can be scaled in accordance with the determined amount to scale the image, and thereafter cropped to fit within the boundaries of the container.