System and method embodiments are provided for adaptive bandwidth management for transporting rendered graphics in a virtual desktop infrastructure. The embodiments include a video detection algorithm that enables a driver for a graphics processor at the server's end to identify displayed video data at the user'send. The driver identifies video traffic that is sent to the user device by tracking repeated bitmap operations at the same display area or block of an application displayed window at the user device. When the number of bitmap operations reaches a pre-determined threshold, then the corresponding area is considered a video display area. When detecting that a bandwidth threshold for transporting graphics data from the server to the device is reached, the data corresponding to the detected video area is compressed using a higher compression rate, thus increasing the loss rate of transferred video data to improve playback experience for the user.