Systems and methods are provided for dynamically reallocating resources during run-time execution of workloads in a distributed accelerator-as-a-service computing system to increase workload execution performance and resource utilization. A workload is executed in the distributed accelerator-as-a-service computing system using an initial set of resources allocated to the executing workload, wherein the allocated resources include accelerator resources (e.g., physical and/or virtual accelerator resources). The performance of the executing workload is monitored to detect a bottleneck condition which causes a decrease in the performance of the executing workload. In response to detecting the bottleneck condition, another set of resources is reallocated to the executing workload, which is determined to reduce or eliminate the bottleneck condition. A live migration process is performed to move the executing workload to the reallocated set of resources such that the workload execution continues using the reallocated set of resources.