Storage and synchronization of metadata in a distributed storage system

    公开(公告)号:US11468015B2

    公开(公告)日:2022-10-11

    申请号:US16919712

    申请日:2020-07-02

    Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triplets corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triplets to create a new version of the SST file. The synchronization is bi-directional.

    Synchronization of metadata in a distributed storage system

    公开(公告)号:US11455280B2

    公开(公告)日:2022-09-27

    申请号:US16919721

    申请日:2020-07-02

    Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

    SYNCHRONIZATION OF METADATA IN A DISTRIBUTED STORAGE SYSTEM

    公开(公告)号:US20220222214A1

    公开(公告)日:2022-07-14

    申请号:US17708312

    申请日:2022-03-30

    Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file, then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

    Global de-duplication of virtual disks in a storage platform

    公开(公告)号:US11314458B2

    公开(公告)日:2022-04-26

    申请号:US17028164

    申请日:2020-09-22

    Abstract: In order to avoid writing duplicates of blocks of data into a storage platform, any virtual disk within the storage platform may have a de-duplication feature enabled. Or, all virtual disks have this feature enabled. For virtual disks with de-duplication enabled, a unique message digest is calculated for every block of data written to that virtual disk. Upon a write, these message digests are consulted in order to determine if a particular block of data has already been written, if so, it is not written again, and if not, it is written. All de-duplication virtual disks are written to a single system virtual disk within the storage platform. De-duplication occurs over the entire storage platform and over all its virtual disks because all message digests are consulted before a write is performed for any virtual disk. A read for a de-duplication virtual desk reads from the system virtual disk.

    STORAGE AND SYNCHRONIZATION OF METADATA IN A DISTRIBUTED STORAGE SYSTEM

    公开(公告)号:US20200334206A1

    公开(公告)日:2020-10-22

    申请号:US16919712

    申请日:2020-07-02

    Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triplets corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triplets to create a new version of the SST file. The synchronization is bi-directional.

    SYNCHRONIZING METADATA IN A DATA STORAGE PLATFORM COMPRISING MULTIPLE COMPUTER NODES

    公开(公告)号:US20200334205A1

    公开(公告)日:2020-10-22

    申请号:US16919630

    申请日:2020-07-02

    Abstract: A client machine writes to a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triplets corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triplets to create a new version of the SST file. The synchronization is bi-directional as between distinct computer nodes.

    Synchronization of metadata in a distributed storage system

    公开(公告)号:US10740300B1

    公开(公告)日:2020-08-11

    申请号:US15834921

    申请日:2017-12-07

    Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform using a storage protocol. Metadata is generated and is stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them in the SST file is compacted with the missing triples to create a new version of the SST file. The other fingerprint file is then analyzed the same way.

Patent Agency Ranking