公开(公告)号：US11500821B2

公开(公告)日：2022-11-15

申请号：US16919630

申请日：2020-07-02

Applicant: Commvault Systems, Inc.

Inventor： Avinash Lakshman ,  Lasaro Camargos ,  Deepak Jain

IPC: G06F7/00 ,  G06F17/00 ,  G06F16/178 ,  G06F11/14 ,  G06F16/13 ,  G06F16/2458 ,  G06F16/2457 ,  G06F16/182 ,  G06F16/27

Abstract: A client machine writes to a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triplets corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triplets to create a new version of the SST file. The synchronization is bi-directional as between distinct computer nodes.

公开(公告)号：US11789830B2

公开(公告)日：2023-10-17

申请号：US17465722

申请日：2021-09-02

Applicant: Commvault Systems, Inc.

Inventor： Deepak Jain ,  Lásaro Camargos ,  Srinivas Lakshman ,  Avinash Lakshman

IPC: G06F11/20 ,  G06F11/14

CPC classification number: G06F11/2058 ,  G06F11/1435 ,  G06F11/1464 ,  G06F11/1469 ,  G06F11/2082

Abstract: A strongly consistent distributed data storage system comprises an enhanced metadata service that is capable of fully recovering all metadata that goes missing when a metadata-carrying disk, disks, and/or partition fail. An illustrative recovery service runs automatically or on demand to bring the metadata node back into full service. Advantages of the recovery service include guaranteed full recovery of all missing metadata, including metadata still residing in commit logs, without impacting strong consistency guarantees of the metadata. The recovery service is network-traffic efficient. In preferred embodiments, the recovery service avoids metadata service downtime at the metadata node, thereby reducing the impact of metadata disk failure on the availability of the system. The disclosed metadata recovery techniques are said to be “self-healing” as they do not need manual intervention and instead automatically detect failures and automatically recover from the failures in a non-disruptive manner.

公开(公告)号：US11647075B2

公开(公告)日：2023-05-09

申请号：US17465683

申请日：2021-09-02

Applicant: Commvault Systems, Inc.

Inventor： Lásaro Camargos ,  Deepak Jain ,  Avinash Lakshman ,  Bharat Pundalik Naik

IPC: G06F15/173 ,  H04L67/104 ,  G06F9/455 ,  G06F11/14 ,  G06F16/182 ,  G06F16/23

CPC classification number: H04L67/1046 ,  G06F9/45558 ,  G06F11/1425 ,  G06F16/182 ,  G06F16/2365 ,  H04L67/1048 ,  G06F2009/45583

Abstract: In a running distributed data storage system that actively processes I/Os, metadata nodes are commissioned and decommissioned without taking down the storage system and without introducing interruptions to metadata or payload data I/O. The inflow of reads and writes continues without interruption even while new metadata nodes are in the process of being added and/or removed and the strong consistency of the system is guaranteed. Commissioning and decommissioning nodes within the running system enables streamlined replacement of permanently failed nodes and advantageously enables the system to adapt elastically to workload changes. An illustrative distributed barrier logic (the “view change barrier”) controls a multi-state process that controls a coordinated step-wise progression of the metadata nodes from an old view to a new normal. Rules for I/O handling govern each state until the state machine loop has been traversed and the system reaches its new normal.

公开(公告)号：US12294622B2

公开(公告)日：2025-05-06

申请号：US18654088

申请日：2024-05-03

Applicant: Commvault Systems, Inc.

Inventor： Lasaro Camargos ,  Bharat Pundalik Naik ,  Avinash Lakshman ,  Deepak Jain

IPC: G06F15/173 ,  G06F9/455 ,  G06F11/14 ,  G06F16/182 ,  G06F16/23 ,  G06F18/25 ,  G06V10/25 ,  G06V10/34 ,  G06V20/54 ,  G06V20/59 ,  G06V40/10 ,  H04L67/104 ,  H04N7/18 ,  H04N23/56 ,  H04N23/66

Abstract: In a running distributed data storage system that actively processes I/Os, metadata nodes are commissioned and decommissioned without taking down the storage system and without introducing interruptions to metadata or payload data I/O. The inflow of reads and writes continues without interruption even while new metadata nodes are in the process of being added and/or removed and the strong consistency of the system is guaranteed. Commissioning and decommissioning nodes within the running system enables streamlined replacement of permanently failed nodes and advantageously enables the system to adapt elastically to workload changes. An illustrative distributed barrier logic (the “view change barrier”) controls a multi-state process that controls a coordinated step-wise progression of the metadata nodes from an old view to a new normal. Rules for I/O handling govern each state until the state machine loop has been traversed and the system reaches its new normal.

公开(公告)号：US12063270B2

公开(公告)日：2024-08-13

申请号：US18128499

申请日：2023-03-30

Applicant: Commvault Systems, Inc.

Inventor： Lásaro Camargos ,  Deepak Jain ,  Avinash Lakshman ,  Bharat Pundalik Naik

IPC: G06F15/173 ,  G06F9/455 ,  G06F11/14 ,  G06F16/182 ,  G06F16/23 ,  G06F18/25 ,  G06V10/25 ,  G06V10/34 ,  G06V20/54 ,  G06V20/59 ,  G06V40/10 ,  H04L67/104 ,  H04N7/18 ,  H04N23/56 ,  H04N23/66

CPC classification number: H04L67/1046 ,  G06F9/45558 ,  G06F11/1425 ,  G06F16/182 ,  G06F16/2365 ,  G06F18/251 ,  G06V10/25 ,  G06V10/34 ,  G06V20/54 ,  G06V20/59 ,  G06V40/10 ,  G06V40/103 ,  H04L67/1048 ,  H04N7/181 ,  H04N7/188 ,  H04N23/56 ,  H04N23/66 ,  G06F2009/45583 ,  G06V2201/08

Abstract: In a running distributed data storage system that actively processes I/Os, metadata nodes are commissioned and decommissioned without taking down the storage system and without introducing interruptions to metadata or payload data I/O. The inflow of reads and writes continues without interruption even while new metadata nodes are in the process of being added and/or removed and the strong consistency of the system is guaranteed. Commissioning and decommissioning nodes within the running system enables streamlined replacement of permanently failed nodes and advantageously enables the system to adapt elastically to workload changes. An illustrative distributed barrier logic (the “view change barrier”) controls a multi-state process that controls a coordinated step-wise progression of the metadata nodes from an old view to a new normal. Rules for I/O handling govern each state until the state machine loop has been traversed and the system reaches its new normal.

公开(公告)号：US20230308507A1

公开(公告)日：2023-09-28

申请号：US18128499

申请日：2023-03-30

Applicant: Commvault Systems, Inc.

Inventor： Lásaro Camargos ,  Deepak Jain ,  Avinash Lakshman ,  Bharat Pundalik Naik

IPC: H04L67/104 ,  G06F9/455 ,  G06F11/14 ,  G06F16/182 ,  G06F16/23

CPC classification number: H04L67/1046 ,  G06F9/45558 ,  G06F11/1425 ,  G06F16/182 ,  G06F16/2365 ,  H04L67/1048 ,  G06F2009/45583

Abstract: In a running distributed data storage system that actively processes I/Os, metadata nodes are commissioned and decommissioned without taking down the storage system and without introducing interruptions to metadata or payload data I/O. The inflow of reads and writes continues without interruption even while new metadata nodes are in the process of being added and/or removed and the strong consistency of the system is guaranteed. Commissioning and decommissioning nodes within the running system enables streamlined replacement of permanently failed nodes and advantageously enables the system to adapt elastically to workload changes. An illustrative distributed barrier logic (the “view change barrier”) controls a multi-state process that controls a coordinated step-wise progression of the metadata nodes from an old view to a new normal. Rules for I/O handling govern each state until the state machine loop has been traversed and the system reaches its new normal.

公开(公告)号：US11468015B2

公开(公告)日：2022-10-11

申请号：US16919712

申请日：2020-07-02

Applicant: Commvault Systems, Inc.

Inventor： Avinash Lakshman ,  Lasaro Camargos ,  Deepak Jain

IPC: G06F7/00 ,  G06F17/00 ,  G06F16/178 ,  G06F11/14 ,  G06F16/13 ,  G06F16/2458 ,  G06F16/2457 ,  G06F16/182 ,  G06F16/27

Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triplets corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triplets to create a new version of the SST file. The synchronization is bi-directional.

公开(公告)号：US11455280B2

公开(公告)日：2022-09-27

申请号：US16919721

申请日：2020-07-02

Applicant: Commvault Systems, Inc.

Inventor： Avinash Lakshman ,  Lasaro Camargos ,  Deepak Jain

IPC: G06F7/00 ,  G06F17/00 ,  G06F16/178 ,  G06F11/14 ,  G06F16/13 ,  G06F16/2458 ,  G06F16/2457 ,  G06F16/182 ,  G06F16/27

Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

公开(公告)号：US20220222214A1

公开(公告)日：2022-07-14

申请号：US17708312

申请日：2022-03-30

Applicant: Commvault Systems, Inc.

Inventor： Avinash Lakshman ,  Lasaro Camargos ,  Deepak Jain

IPC: G06F16/178 ,  G06F11/14 ,  G06F16/13 ,  G06F16/2458 ,  G06F16/2457

Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file, then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

公开(公告)号：US10740300B1

公开(公告)日：2020-08-11

申请号：US15834921

申请日：2017-12-07

Applicant: Commvault Systems, Inc.

Inventor： Avinash Lakshman ,  Lasaro Camargos ,  Deepak Jain

IPC: G06F7/00 ,  G06F17/00 ,  G06F16/178 ,  G06F11/14 ,  G06F16/13 ,  G06F16/2458 ,  G06F16/2457 ,  G06F16/182 ,  G06F16/27

Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform using a storage protocol. Metadata is generated and is stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them in the SST file is compacted with the missing triples to create a new version of the SST file. The other fingerprint file is then analyzed the same way.