公开(公告)号：US11768739B2

公开(公告)日：2023-09-26

申请号：US16943674

申请日：2020-07-30

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F16/27 ,  G06F11/14

CPC classification number: G06F11/1464 ,  G06F16/27 ,  G06F11/1456 ,  G06F2201/84

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

公开(公告)号：US12007846B2

公开(公告)日：2024-06-11

申请号：US16943776

申请日：2020-07-30

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F16/30 ,  G06F11/14 ,  G06F16/27

CPC classification number: G06F11/1464 ,  G06F16/27 ,  G06F11/1456 ,  G06F2201/84

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

公开(公告)号：US20170262348A1

公开(公告)日：2017-09-14

申请号：US15605205

申请日：2017-05-25

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F11/14 ,  G06F17/30

CPC classification number: G06F11/1464 ,  G06F11/1456 ,  G06F16/27 ,  G06F2201/84

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

公开(公告)号：US10776217B2

公开(公告)日：2020-09-15

申请号：US15605205

申请日：2017-05-25

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F7/00 ,  G06F17/00 ,  G06F11/14 ,  G06F16/27

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

公开(公告)号：US09690671B2

公开(公告)日：2017-06-27

申请号：US14527563

申请日：2014-10-29

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F7/00 ,  G06F17/00 ,  G06F11/14 ,  G06F17/30

CPC classification number: G06F11/1464 ,  G06F11/1456 ,  G06F17/30575 ,  G06F2201/84

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

公开(公告)号：US20150127608A1

公开(公告)日：2015-05-07

申请号：US14527563

申请日：2014-10-29

Applicant: Cloudera, Inc.

Inventor： Jonathan Ming-Cyn Hsieh ,  Matteo Bertozzi

IPC: G06F17/30 ,  G06F11/14

CPC classification number: G06F11/1464 ,  G06F11/1456 ,  G06F17/30575 ,  G06F2201/84

Abstract: Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

Abstract translation: 本文提供了可扩展架构，系统和服务，用于在分布式计算环境中创建基于清单的快照。 在一些实施例中，响应于接收到创建数据对象的快照的请求，主节点识别数据对象在其中存储在云计算平台中的多个从节点，并且创建表示数据对象的快照的快照清单 。 快照清单包括包含快照清单中的多个文件名的列表的文件以及用于在分布式数据库系统中定位多个文件的参考信息。 可以创建快照，而不会中断由主节点指导的各种区域服务器的在线模式的I / O操作。 此外，还公开了创建快照的日志滚动方法，其中标记了日志文件。 日志条目的重放可以减少快照中因果一致性的概率。