The present disclosure provides systems, methods, and computer-readable media for managing graphics processing unit (GPU) allocation for a virtual machine (VM). A first GPU driver, associated with a first GPU, is offloaded from an operating system (OS) of the VM. Then, the first GPU is deallocated from the VM. A second GPU is allocated to the VM, and a second GPU driver, associated with the second GPU, is loaded in the OS of the VM. To restore a GPU context from the first GPU within the second GPU, a GPU command log from the first GPU is replayed to the second GPU.