A method for creating an optimized gradient mesh of a vector-based image from a raster-based image. In one implementation, a set of boundaries for an object on a raster-based image may be received. An initial gradient mesh of the object may be created. A residual energy between the object on the raster-based image and a rendered initial gradient mesh may be minimized to generate an optimized gradient mesh.