Methods of optimized video transfer are described in which display data is divided into a plurality of portions, each portion corresponding to a class of image data, and the different portions are encoded for transmission to a remote display device using different encoding schemes. In an embodiment, the composite display data is never rendered at the source device and instead portions of display data are generated by an application based on an advertised set of classes of image data. In other embodiments, the composite display data is stored in a framebuffer at the source device and is split into portions according to a classification of the display data. In various embodiments the classification may be performed based on sniffing graphics commands used to generate the composite display data and in other embodiments, the classification may be performed based on a rate of change of pixels in the composite display data.