Abstract:
The technique introduced here includes a system and method for identification of duplicate data directly at a data-object level. The technique illustratively utilizes a hierarchical tree of fingerprints for each data object to compare data objects and identify duplicate data blocks referenced by the data objects. The hierarchical fingerprint trees are constructed in such a manner that a top-level fingerprint (or object-level fingerprint) of the hierarchical tree is representative of all data blocks referenced by a storage system. In embodiments, inline techniques are utilized to generate hierarchical fingerprints for new data objects as they are created, and an object-level fingerprint of the new data object is compared against preexisting object-level fingerprints in the storage system to identify exact or close matches. While exact matches result in complete deduplication of data blocks referenced by the data object, hierarchical comparison methods are used for identifying and mapping duplicate data blocks referenced by closely related data objects.
Abstract:
A change in workload characteristics detected at one tier of a multi-tiered cache is communicated to another tier of the multi-tiered cache. Multiple caching elements exist at different tiers, and at least one tier includes a cache element that is dynamically resizable. The communicated change in workload characteristics causes the receiving tier to adjust at least one aspect of cache performance in the multi-tiered cache. In one aspect, at least one dynamically resizable element in the multi-tiered cache is resized responsive to the change in workload characteristics.
Abstract:
A storage area network (SAN)-attached storage system architecture is disclosed. The storage system provides strongly consistent distributed storage communication protocol semantics, such as SCSI target semantics. The system includes a mechanism for presenting a single distributed logical unit, comprising one or more logical sub-units, as a single logical unit of storage to a host system by associating each of the logical sub-units that make up the single distributed logical unit with a single host visible identifier that corresponds to the single distributed logical unit. The system further includes mechanisms to maintain consistent context information for each of the logical sub-units such that the logical sub-units are not visible to a host system as separate entities from the single distributed logical unit.
Abstract:
System and method for retaining deduplication of data blocks of a resulting storage object (e.g., a flexible volume) from a split operation of a clone of a base storage object. The clone may comprise data blocks that are shared with at least one data block of the base storage object and at least one data block that is not shared with at least one data block of the base storage object. The data blocks of the clone that are shared with the base storage object may be indicated to receive a write allocation that may comprise assigning a new pointer to a indicated data block. Each data block may comprise a plurality of pointers comprising a virtual address pointer and a physical address pointer. As such, data blocks of the clone comprising the same virtual address pointer may be assigned a single physical address pointer.
Abstract:
System and method for migrating data from a source storage site to a destination storage site. The data may be comprised within storage objects (e.g., flexible volumes). A base storage object may comprise a parent storage object and a storage object clone may comprise a storage object that is derived from the base storage object. As such, a hierarchical relationship exists between the base storage object and the storage object clone. The storage object clone may comprise a writable point-in-time image of the parent storage object. If a migration of the base storage object and the storage object clone is performed, then the hierarchical relationship between the base storage object and the storage object clone is retained after the storage objects are migrated from the source storage site to the destination storage site. As such, the system and method for migrating data may enable storage space and network bandwidth savings.
Abstract:
Methods and systems for efficiently determining a similarity between two or more datasets. In one embodiment, the similarity is determined based on comparing a subset of sorted frequency-weighted blocks from one dataset to a subset of sorted frequency-weighed blocks from another dataset. Data blocks of a dataset are converted into hash values that are frequency-weighted. These frequency-weighted hash values can be compared to frequency-weighted hash values of another dataset to determine a similarity of the two datasets. In another embodiment, upon a change of a block in a subset of the dataset, the similarity value is re-determined without resorting or hashing the blocks of a dataset other than the blocks of the subset, resulting in an increased performance of a similarity comparison. In another embodiment, blocks of a dataset are excluded based on a block-filtering rule to increase the accuracy of the similarity comparison.
Abstract:
Fault isolation capabilities made available by user space can be provided for a embedded network storage system without sacrificing efficiency. By giving user space processes direct access to specific devices (e.g., network interface cards and storage adapters), processes in a user space can initiate Input/Output requests without issuing system calls (and entering kernel mode). The multiple user spaces processes can initiate requests serviced by a user space device driver by sharing a read-only address space that maps the entire physical memory one-to-one. In addition, a user space process can initiate communication with another user space process by use of transmit and receive queues similar to transmit and receive queues used by hardware devices. And, a mechanism of ensuring that virtual addresses that work in one address space reference the same physical page in another address space is used.
Abstract:
A system and method are described for aggregating management of devices connected to a server. In one embodiment, devices connected to a server may be identified by an operating system of the server. The devices may be in communication with one another through a network. The operating system may communicate a network identifier of each device to a service processor of the server. The service processor may aggregately manage the devices independent from the operating system. The service processor may receive management commands from an external source. Each of the management commands may be associated with one of the connected devices, but the connected devices may be incapable of communicating directly with the external source. The service processor may communicate each management command to the device associated with the management command using the network identifier of the associated device. The management command can then be performed on the associated device.
Abstract:
A technique for organizing data to facilitate data deduplication includes dividing a block-based set of data into multiple "chunks", where the chunk boundaries are independent of the block boundaries (due to the hashing algorithm). Metadata of the data set, such as block pointers for locating the data, are stored in a tree structure that includes multiple levels, each of which includes at least one node. The lowest level of the tree includes multiple nodes that each contain chunk metadata relating to the chunks of the data set. In each node of the lowest level of the buffer tree, the chunk metadata contained therein identifies at least one of the chunks. The chunks (user-level data) are stored in one or more system files that are separate from the buffer tree and not visible to the user.
Abstract:
A computer storage system includes multiple disk trays, each disk tray holding two or more physical disks. The disks on a single tray are virtualized into a single logical disk. The single logical disk reports to the RAID (redundant array of inexpensive disks) subsystem, creating the impression that there is one large capacity disk. In one implementation, each disk in the tray is allocated to a different RAID group. By allocating the disks in a tray to different RAID groups, if the tray is removed, only a portion of several different RAID groups are removed. This arrangement permits a simple reconstruction of the RAID groups if a disk tray is removed from the system.