A mechanism for memory change tracking during migration of a virtual machine (VM) with VM-controlled assigned peripherals is disclosed. A method of the invention includes receiving a state of a VM-controlled peripheral device running on a destination host machine from an origin host machine, wherein the VM is executed by a hypervisor of the destination host machine, providing the received state of the peripheral device to at least one of the VM, a memory of the VM, or the peripheral device in order to cause the peripheral device to be restored to that state on the destination host machine, and interacting with the VM in order to cause the VM to start the peripheral device in the restored state.