Abstract:
Systems and methods for reducing metadata in a write-anywhere storage system are disclosed herein. The system includes a plurality of clients coupled with a plurality of storage nodes, each storage node having a plurality of primary storage devices coupled thereto. A memory management unit including cache memory is included in the client. The memory management unit serves as a cache for data produced by the clients before the data is stored in the primary storage. The cache includes an extent cache, an extent index, a commit cache and a commit index. The movement of data and metadata is by an interval tree. Methods for reducing data in the interval tree increase data storage and data retrieval performance of the system.
Abstract:
A data migration system supports a low-latency and reduced overhead data storage protocol for data storage sharing in a non-collision fashion which does not require inter-communication and permanent arbitration between data storage controllers to decide on the data placement/routing. The multiple data fragments of data sets are prevented from routing to the same storage devices by a multi-step selection protocol which selects (in a first phase of the selection routine) a healthy highest ranked drive enclosure, and further selects (in a second phase of the selection routine) a healthy highest-ranked data storage controller residing in the selected drive enclosure, for routing data fragments to different storage pools assigned to the selected data storage devices for exclusive “writing” and data modification. The selection protocol also contemplates various failure scenarios in a data placement collision free manner.
Abstract:
Variable Redundancy Distributed (VRD) RAID controller in a data storage environment contains embedded RAID logic permitting to choose and compute a desired redundancy coding scheme from a plurality thereof pre-programmed and embedded in a Compute Engine in the VRD RAID controller. “Write” or “Read” requests which are received from data generating entities, contain information identifying a type of the redundancy coding scheme of interest. The controller decodes the request, and automatically applies the desired computation to the incoming data without burdening the CPU with the computational activity. The variable redundancy computational ability of the subject systems provides an extremely versatile and flexible tool for RAID operations.
Abstract:
Method and system for data migration between data generating entities and data storage devices protected by de-clustered RAID algorithm are enhanced by dynamically controlling the I/O activity towards the data storage devices (NVM devices) based on their remaining lifespan (health) with the goal to prevent multiple devices selected for writing a parity stripe information from simultaneous failures. This feature is rendered by polling the remaining health of NVM devices in the RAID pool, computing a weighted lifespan for each NVM device, comparing the latter to an average of all NVM devices in the pool, and adjusting the I/O activity towards the NVM device of interest accordingly. If the weighted lifespan exceeds the average lifespan in the pool, the allowed I/O activity is increased, and if the weighted lifespan is below the average for the pool, then the device in question is sent less “writes”.
Abstract:
Systems and methods for reducing metadata in a write-anywhere storage system are disclosed herein. The system includes a plurality of clients coupled with a plurality of storage nodes, each storage node having a plurality of primary storage devices coupled thereto. A memory management unit including cache memory is included in the client. The memory management unit serves as a cache for data produced by the clients before the data is stored in the primary storage. The cache includes an extent cache, an extent index, a commit cache and a commit index. The movement of data and metadata is by an interval tree. Methods for reducing data in the interval tree increase data storage and data retrieval performance of the system.
Abstract:
Systems and methods for reducing metadata in a write-anywhere storage system are disclosed herein. The system includes a plurality of clients coupled with a plurality of storage nodes, each storage node having a plurality of primary storage devices coupled thereto. A memory management unit including cache memory is included in the client. The memory management unit serves as a cache for data produced by the clients before the data is stored in the primary storage. The cache includes an extent cache, an extent index, a commit cache and a commit index. The movement of data and metadata is by an interval tree. Methods for reducing data in the interval tree increase data storage and data retrieval performance of the system.
Abstract:
Distributed Compute Engine (DCE) memory controller in a data storage environment contains embedded logic and arithmetic functionality for Boolean logical and arithmetic operations. “Write” or “Read” requests which are received from data generating entities, contain a Physical Address field identifying an address of a data block to be retrieved from the external memory, and a Control bits field identifying a type of computational operation to be performed. The DCE memory controller decodes the request, and applies the desired compute operation autonomically to the contents of an external memory and/or the incoming data without burdening the CPU with the computational activity.
Abstract:
The system and routine for data caching leverages the properties of Network-Attached Non-Volatile Memories (NANVMs) to provide virtualized secure node-local storage services to the network users with reduced data movement across the NANVMs. The caching routine reserves storage resources (storage partitions) on NANVM devices, migrates data required for the target application execution to the allocated storage partitions, and directs the network clients to dynamically “mount” to the storage partitions based on application data requirements. Only those clients and applications that present valid credentials and satisfactory computing capabilities can access the data in the specific storage partitions. Several clients can have an access to the same storage partitions without duplication or replicating the data. A Global Data Indexing sub-system supports the efficient operation of the subject system. The Global Data Indexing Sub-System provides mapping between the storage partitions, data sets, applications, client nodes, as well as their credentials/capabilities.
Abstract:
In the data storage system the storage area network performs XOR operations on incoming data for parity generation without buffering data through a centralized RAID engine or processor. The hardware for calculating the XOR data is distributed to incrementally calculate data parity in parallel across each data channel and may be implemented as a set of FPGAs with low bandwidths to efficiently scale as the amount of storage memory increases. A host adaptively appoints data storage controllers in the storage area network to perform XOR parity operations on data passing therethrough. The system provides data migration and parity generation in a simple and effective matter and attains a reduction in cost and power consumption.
Abstract:
The present invention is directed to data migration, and particularly, Parity Group migration, between high performance data generating entities and data storage structure in which distributed NVM arrays are used as a single intermediate logical storage which requires a global registry/addressing capability that facilitates the storage and retrieval of the locality information (metadata) for any given fragment of unstructured data and where Parity Group Identifier and Parity Group Information (PGI) descriptors for the Parity Groups' members tracking, are created and distributed in the intermediate distributed NVM arrays as a part of the non-deterministic data addressing system to ensure coherency and fault tolerance for the data and the metadata. The PGI descriptors act as collection points for state describing the residency and replay status of members of the Parity Groups.