Offload processing may be provided that is not dedicated to a primary processor or a subset of primary processors. A system may have one or more offload processors, for example, GPUs, coupled to data storage slots of the system, which can be shared by multiple primary processors of the system. The offload processor(s) may be housed within a device configured to be coupled to a storage slot, for example, as if the device were a storage drive. The one or more offload processors may be housed within a device that includes an interface in conformance with a version of an NVMe specification and may have a form factor in accordance with the U.2 specification. Offload processing devices may be communicatively coupled to one or more primary processors by switching fabric disposed between the one or more primary processors and the storage slot to which the offload processing device is connected.