Abstract:
The methods and systems for organizing and managing a file system on top of a content addressable object store are provided. User content is associated with a record including a content hash associated with an address in the content addressable object store. The content hash is a function of user content and determined by dividing the file into data objects, generating a pointer tree of hashes on top of the data objects, and calculating a hash of the root of the pointer tree. The record, the pointer tree, and the file are stored in the object store. Reading user content from the object store includes reading the record, fetching the content hash to find a root of the pointer tree, descending leaves of the pointer tree to read the hashes of the data objects associated with user content, and reassembling the data objects in sequential file data.
Abstract:
Computer-implemented methods and systems for managing data objects within a computer network infrastructure that facilitate more efficient and reliable data storage and access are provided. An exemplary method may comprise establishing a physical identifier for each storage resource. A plurality of unique virtual identifiers for each storage resource is generated based on the physical identifier. The plurality of unique virtual identifiers is stored in a consistent ring namespace accessible to every node. An object identifier associated with a location of a data object to be stored in a storage resource is generated. The object identifier is mapped to the consistent ring namespace. The method may also include enabling traversing the consistent ring namespace from any node in the computer network to locate and access the data object.
Abstract:
The methods and systems for organizing and managing a file system on top of a content addressable object store are provided. User content is associated with a record including a content hash associated with an address in the content addressable object store. The content hash is a function of user content and determined by dividing the file into data objects, generating a pointer tree of hashes on top of the data objects, and calculating a hash of the root of the pointer tree. The record, the pointer tree, and the file are stored in the object store. Reading user content from the object store includes reading the record, fetching the content hash to find a root of the pointer tree, descending leaves of the pointer tree to read the hashes of the data objects associated with user content, and reassembling the data objects in sequential file data.
Abstract:
Disclosed is a file versioning system and corresponding methods for its operation. The file versioning system allows making snapshots of the file system every time there is a modification to the file system or its items. The snapshots may be linked to their immediate predecessors. Some older snapshots may be discarded according to a “thinning out” process based on multiple criteria. The snapshots may be displayed to a user in a manner making it easy to select a desired version.
Abstract:
Disclosed is the technology for data scrubbing in a cluster-based storage system. This technology allows protecting data against failures of storage devices by periodically reading data object replicas and data object hashes stored in a plurality of storage devices and rewriting those data object replicas that have errors. The present disclosure addresses aspects of writing data object replicas and hashes, checking validity of data object replicas, and performing data scrubbing based upon results of the checking.