Abstract:
A technique for eliminating duplicate data is provided. Upon receipt of a new data set, one or more anchor points are identified within the data set. A bit-by-bit data comparison is then performed of the region surrounding the anchor point in the received data set with the region surrounding an anchor point stored within a pattern database to identify forward/backward delta values. The duplicate data identified by the anchor point, forward and backward delta values is then replaced in the received data set with a storage indicator.
Abstract:
A sampling based technique for eliminating duplicate data (de-duplication) stored on storage resources, is provided. According to the invention, when a new data set, e.g., a backup data stream, is received by a server, e.g., a storage system or virtual tape library (VTL) system implementing the invention, one or more anchors are identified within the new data set. The anchors are identified using a novel anchor detection circuitry in accordance with an illustrative embodiment of the present invention. Upon receipt of the new data set by, for example, a network adapter of a VTL system, the data set is transferred using direct memory access (DMA) operations to a memory associated with an anchor detection hardware card that is operatively interconnected with the storage system. The anchor detection hardware card may be implemented as, for example, a FPGA is to quickly identify anchors within the data set. As the anchor detection process is performed using a hardware assist, the load on a main processor of the system is reduced, thereby enabling line speed de-duplication.
Abstract:
A method and an apparatus to store data patterns are presented. In one embodiment, the method includes searching a pattern repository to find prior copies of a pattern and to reference one of the prior copies, or insert a new copy, based on the access time of the prior copy and the effect on the sequential stream performance.
Abstract:
According to one or more embodiments of the present invention, a network cache intercepts data requested by a client from a remote server interconnected with the cache through one or more wide area network (WAN) links (e.g., for Wide Area File Services, or 'WAFS'). The network cache stores the data and sends the data to the client. The cache may then intercept a first write request for the data from the client to the remote server, and determine one or more portions of the data in the write request that changed from the data stored at the cache (e.g., according to one or more hashes created based on the data). The network cache then sends a second write request for only the changed portions of the data to the remote server.
Abstract:
A method and an apparatus to store data patterns are presented. In one embodiment, the method includes searching a pattern repository to find prior copies of a pattern and to reference one of the prior copies, or insert a new copy, based on the access time of the prior copy and the effect on the sequential stream performance.
Abstract:
A technique for eliminating duplicate data is provided. Upon receipt of a new data set, one or more anchor points are identified within the data set. A bit-by-bit data comparison is then performed of the region surrounding the anchor point in the received data set with the region surrounding an anchor point stored within a pattern database to identify forward/backward delta values. The duplicate data identified by the anchor point, forward and backward delta values is then replaced in the received data set with a storage indicator.