Embodiments herein describe techniques for executing VMs on hosts that include an accelerator. The hosts can use the accelerators to perform specialized tasks such as floating-point arithmetic, encryption, image processing, etc. Moreover, VMs can be migrated between hosts. To do so, the state of the processor is saved on the current host thereby saving the state of the VM. For example, by saving the processor state, once the data corresponding to the VM is loaded into a destination host, the processor can be initialized to the saved state in order to resume the VM. In addition to saving the processor state, the embodiments herein save the state of the accelerator on a FPGA. That is, unlike previous systems where tasks executed by the accelerator are discarded when migrating the VM, the state of the accelerator can be saved and used to initialize an FPGA accelerator in the destination host.