Described herein are devices, systems and methods to utilize non-volatile memory to save and retrieve data that is used to accelerate the load and resume of GPU accelerated applications. Non-volatile memory and GPU logic are configured to enable the GPU to directly access the non-volatile memory to enable data to be read without requiring the data to traverse the CPU and CPU memory. This data access path creates a faster method for loading data into GPU local memory.