A queued, byte addressed system and method for accessing flash memory and other non-volatile storage class memory, and potentially other types of non-volatile memory (NVM) storage systems. In a host device, e.g., a standalone or networked computer, having attached NVM device storage integrated into a switching fabric wherein the NVM device appears as an industry standard OFED™ RDMA verbs provider. The verbs provider enables communicating with a ‘local storage peer’ using the existing OpenFabrics RDMA host functionality. User applications issue RDMA Read/Write directives to the ‘local peer (seen as a persistent storage) in NVM enabling NVM memory access at byte granularity. The queued, byte addressed system and method provides for Zero copy NVM access. The methods enables operations that establish application private Queue Pairs to provide asynchronous NVM memory access operations at byte level granularity.