Systems and methods for offloading processing from a host to one or more storage processing units using an interconnect network are provided. One such method includes receiving a processing task from the host at a first storage processing unit (SPU) of a plurality of SPUs via a host interface, performing, at the first SPU, the processing task, and transferring data from the first SPU to a second SPU via an interconnection network, where each of the plurality of SPUs includes a non-volatile memory (NVM) and a processing circuitry configured to perform the processing task.