An apparatus to facilitate metrics and security-based accelerator service rescheduling and auto-scaling using a programmable network device is disclosed. The apparatus includes processors to collect metrics corresponding to communication links between microservices of a service managed by a service mesh; determine, based on analysis of the metrics, that a workload of the service can be accelerated by offload to a hardware accelerator device; generate a scaling request to cause the hardware accelerator device to be allocated to a cluster of hardware devices configured for the service; cause the scaling request to be transmitted to a programmable network device managing the hardware accelerator device, the programmable network device to allocate the hardware accelerator device to the cluster and to register the hardware accelerator device with the service mesh; and schedule the workload of the service to the hardware accelerator device.