A novel and useful system and method of improving the utilization of a special purpose accelerator in a system incorporating a general purpose processor. In some embodiments, the current queue status of the special purpose accelerator is periodically monitored using a background monitoring process/thread and the current queue status is stored in a shared memory. A shim redirection layer added a priori to a library function task determines at runtime and in user space whether to execute the library function task on the special purpose accelerator or the general purpose processor. At runtime, using the shim redirection layer and based on the current queue status, it is determined whether to execute the library function task on the special purpose accelerator or on the general purpose processor.