公开(公告)号：US11886284B2

公开(公告)日：2024-01-30

申请号：US17746841

申请日：2022-05-17

Applicant: Futurewei Technologies, Inc.

Inventor： Le Cai ,  QingQing Zhou ,  Yang Sun

IPC: G06F11/07 ,  G06F16/11 ,  G06F16/25 ,  G06F16/27 ,  G06F11/14

CPC classification number: G06F11/0793 ,  G06F11/079 ,  G06F11/0709 ,  G06F11/0751 ,  G06F11/0787 ,  G06F11/1451 ,  G06F11/1464 ,  G06F11/1469 ,  G06F16/128 ,  G06F16/258 ,  G06F16/27 ,  G06F2201/80 ,  G06F2201/84

Abstract: A method for data redistribution of a job data in a first datanode (DN) to at least one additional DN in a Massively Parallel Processing (MPP) Database (DB) is provided. The method includes recording a snapshot of the job data, creating a first data portion in the first DN and a redistribution data portion in the first DN, collecting changes to a job data copy stored in a temporary table, and initiating transfer of the redistribution data portion to the at least one additional DN.

公开(公告)号：US09576026B2

公开(公告)日：2017-02-21

申请号：US13802023

申请日：2013-03-13

Applicant: FutureWei Technologies, Inc.

Inventor： Mohan Singamshetty ,  Yang Sun ,  Qingqing Zhou

IPC: G06F17/30

CPC classification number: G06F17/30445

Abstract: System and method embodiments are provided for improving the performance of query processing in a massively parallel processing (MPP) database system by pushing down join query processing to data nodes recursively. An embodiment method includes receiving, at a coordinator process, a join query associated with a plurality of tables of the MPP database system, generating, at the coordinator process, an execution plan tree for the join query, and processing, at each of a plurality of data nodes communicating with the coordinator process, the execution plan tree to obtain join query results. The method further includes, upon detecting a next join operator below a top join operator in the execution plan tree at each of the data nodes, forwarding to the other data nodes a sub-tree for the next join operator, and receiving, at each of the data nodes from the other data nodes, sub-tree processing results.

Abstract translation: 提供了系统和方法实施例，用于通过将连接查询处理递归地推送到数据节点来提高大规模并行处理（MPP）数据库系统中查询处理的性能。 一种实施方式包括在协调器进程处接收与MPP数据库系统的多个表相关联的连接查询，在协调器进程处生成用于连接查询的执行计划树，并在多个表中的每一个处处理 的数据节点与协调器进程通信，执行计划树获取连接查询结果。 该方法还包括：在每个数据节点处检测执行计划树中的顶部连接运算符下面的下一个连接运算符，向下一个连接运算符转发另一个数据节点的子树，并在每个 来自其他数据节点的数据节点，子树处理结果。

公开(公告)号：US09239741B2

公开(公告)日：2016-01-19

申请号：US13653308

申请日：2012-10-16

Applicant: FutureWei Technologies, Inc.

Inventor： Yuan Dai ,  Siew Sim-Tang ,  Kamini Jagtiani ,  Yang Sun ,  Gangavara Prasad Varakur ,  Bai Yang

IPC: G06F9/50 ,  G06F17/30

CPC classification number: G06F9/5061 ,  G06F17/30312 ,  G06F17/30445 ,  G06F17/30557 ,  G06F2209/505

Abstract: An embodiment method for massively parallel processing includes initiating a management instance on an initial machine, the management instance generating an initial partition corresponding to the initial machine, determining a total number of partitions desired for processing a database, the total number of partitions including the initial partition, determining a number of additional machines available to process the database, grouping the initial machine and the additional machines together in a pod, and launching the management instance on the additional machines in the pod to generate the total number of partitions desired for the database. Additional embodiment methods and an embodiment system operable to perform such methods are also disclosed.

Abstract translation: 用于大规模并行处理的实施例方法包括在初始机器上启动管理实例，所述管理实例生成对应于初始机器的初始分区，确定处理数据库所需的分区的总数，包括初始 分区，确定可用于处理数据库的多个附加机器，将起始机器和附加机器分组到一个pod中，并在该pod中的其他机器上启动管理实例以生成数据库所需的分区总数 。 还公开了另外的实施例方法和可操作以执行这些方法的实施例系统。

公开(公告)号：US09195701B2

公开(公告)日：2015-11-24

申请号：US13663237

申请日：2012-10-29

Applicant: FutureWei Technologies, Inc.

Inventor： Kamini Jagtiani ,  Yang Sun ,  Siew Sim-Tang ,  Gangavara Prasad Varakur

IPC: G06F17/30

CPC classification number: G06F17/30339

Abstract: An embodiment method for massively parallel processing includes assigning a primary key to a first table in a database and a foreign key to a second table in the database, the foreign key of the second table identical to the primary key of the first table, determining a number of partition groups desired for the database, partitioning the first table into first partitions based on the primary key assigned and the number of partition groups desired, partitioning the second table into second partitions based on the foreign key assigned and the number of partition groups desired, and distributing the first partitions and the second partitions to the partition groups as partitioned. An embodiment system for implementing the embodiment methods is also disclosed.

Abstract translation: 用于大规模并行处理的实施例方法包括：将主密钥分配给数据库中的第一表，将外键分配给数据库中的第二表，第二表的外键与第一表的主键相匹配， 数据库所需的分区组数量，基于分配的主键和所分配的分组数将第一分区划分成第一分区，基于所分配的外键划分第二分区到第二分区，以及期望的分组数量 并且将第一分区和第二分区分配给分区的分区组。 还公开了用于实现实施方式的实施例系统。

公开(公告)号：US20200034472A1

公开(公告)日：2020-01-30

申请号：US16047458

申请日：2018-07-27

Applicant: Futurewei Technologies, Inc.

Inventor： Le Cai ,  Jianjun Chen ,  Yang Sun ,  Xiaolin Du ,  Jun Chen ,  Yu Chen ,  Chunfeng Pei

IPC: G06F17/30

Abstract: A database system comprises a persistent storage device, a log node including a memory and a processor, and a plurality of database nodes. A database node includes a cache memory configured to store a database instance, and a processor configured to initiate a database transaction by sending a snapshot request to the log node, the snapshot request including a list of pages that were either replaced or newly loaded in the cache memory. The log node processor is configured to send a snapshot response to the database node, wherein the snapshot response includes a snapshot of the database and a list of changed pages of the database instances. The database node processor is configured to update the status of the pages in cached memory according to the snapshot response and perform the database transaction.

公开(公告)号：US20140280020A1

公开(公告)日：2014-09-18

申请号：US13802023

申请日：2013-03-13

Applicant: FUTUREWEI TECHNOLOGIES, INC.

Inventor： Mohan Singamshetty ,  Yang Sun ,  Qingqing Zhou

IPC: G06F17/30

CPC classification number: G06F17/30445

Abstract: System and method embodiments are provided for improving the performance of query processing in a massively parallel processing (MPP) database system by pushing down join query processing to data nodes recursively. An embodiment method includes receiving, at a coordinator process, a join query associated with a plurality of tables of the MPP database system, generating, at the coordinator process, an execution plan tree for the join query, and processing, at each of a plurality of data nodes communicating with the coordinator process, the execution plan tree to obtain join query results. The method further includes, upon detecting a next join operator below a top join operator in the execution plan tree at each of the data nodes, forwarding to the other data nodes a sub-tree for the next join operator, and receiving, at each of the data nodes from the other data nodes, sub-tree processing results.

Abstract translation: 提供了系统和方法实施例，用于通过将连接查询处理递归地推送到数据节点来提高大规模并行处理（MPP）数据库系统中查询处理的性能。 一种实施方式包括在协调器进程处接收与MPP数据库系统的多个表相关联的连接查询，在协调器进程处生成用于连接查询的执行计划树，并在多个表中的每一个处处理 的数据节点与协调器进程通信，执行计划树获取连接查询结果。 该方法还包括：在每个数据节点处检测执行计划树中的顶部连接运算符下面的下一个连接运算符，向下一个连接运算符转发另一个数据节点的子树，并在每个 来自其他数据节点的数据节点，子树处理结果。

公开(公告)号：US08799284B2

公开(公告)日：2014-08-05

申请号：US13691402

申请日：2012-11-30

Applicant: FutureWei Technologies, Inc.

Inventor： Kamini Jagtiani ,  Siew Sim-Tang ,  Yang Sun ,  Gangavara Prasad Varakur

IPC: G06F17/30

CPC classification number: G06F17/30707 ,  G06F17/30445 ,  G06F17/30584

Abstract: A massively parallel processing (MPP) database can be re-partitioned/re-balanced while remaining on-line through a staged migration procedure. Staged migration may include a first stage and a second stage. During the first stage, entries in an existing partition are reallocated to the new partition, and the catalog is updated to associate the re-allocated entries with both the existing partition and the new partition such that queries for the re-allocated entries are directed toward the existing partition and the new partition. During the second stage, the re-allocated entries are migrated from the existing partition to the new partition, and after the migration is complete, the catalog is re-updated to associate the migrated entries with the new partition such that new queries are directed toward the new partition.

Abstract translation: 大规模并行处理（MPP）数据库可以重新分区/重新平衡，同时通过分阶段迁移过程保持在线状态。 分期迁移可以包括第一阶段和第二阶段。 在第一阶段，将现有分区中的条目重新分配到新分区，并更新目录以将重新分配的条目与现有分区和新分区相关联，以使重新分配的条目的查询指向 现有分区和新分区。 在第二阶段，重新分配的条目将从现有分区迁移到新分区，迁移完成后，重新更新目录以将迁移的条目与新分区相关联，以使新查询指向 新的分区。

公开(公告)号：US20140156666A1

公开(公告)日：2014-06-05

申请号：US13691402

申请日：2012-11-30

Applicant: FUTUREWEI TECHNOLOGIES, INC.

Inventor： Kamini Jagtiani ,  Siew Sim-Tang ,  Yang Sun ,  Gangavara Prasad Varakur

IPC: G06F17/30

CPC classification number: G06F17/30707 ,  G06F17/30445 ,  G06F17/30584

Abstract: A massively parallel processing (MPP) database can be re-partitioned/re-balanced while remaining on-line through a staged migration procedure. Staged migration may include a first stage and a second stage. During the first stage, entries in an existing partition are reallocated to the new partition, and the catalog is updated to associate the re-allocated entries with both the existing partition and the new partition such that queries for the re-allocated entries are directed toward the existing partition and the new partition. During the second stage, the re-allocated entries are migrated from the existing partition to the new partition, and after the migration is complete, the catalog is re-updated to associate the migrated entries with the new partition such that new queries are directed toward the new partition.

Abstract translation: 大规模并行处理（MPP）数据库可以重新分区/重新平衡，同时通过分阶段迁移过程保持在线状态。 分期迁移可以包括第一阶段和第二阶段。 在第一阶段，将现有分区中的条目重新分配到新分区，并更新目录以将重新分配的条目与现有分区和新分区相关联，以使重新分配的条目的查询指向 现有分区和新分区。 在第二阶段，重新分配的条目将从现有分区迁移到新分区，迁移完成后，重新更新目录以将迁移的条目与新分区相关联，以使新查询指向 新的分区。

公开(公告)号：US20200210273A1

公开(公告)日：2020-07-02

申请号：US16737534

申请日：2020-01-08

Applicant: Futurewei Technologies, Inc.

Inventor： Le Cai ,  Qingqing Zhou ,  Yang Sun

IPC: G06F11/07 ,  G06F16/27 ,  G06F11/14 ,  G06F16/25 ,  G06F16/11

Abstract: A method for data redistribution of a job data in a first datanode (DN) to at least one additional DN in a Massively Parallel Processing (MPP) Database (DB) is provided. The method includes recording a snapshot of the job data, creating a first data portion in the first DN and a redistribution data portion in the first DN, collecting changes to a job data copy stored in a temporary table, and initiating transfer of the redistribution data portion to the at least one additional DN.

公开(公告)号：US20170097972A1

公开(公告)日：2017-04-06

申请号：US14872811

申请日：2015-10-01

Applicant: Futurewei Technologies, Inc.

Inventor： Qingqing Zhou ,  Yang Sun ,  Guogen Zhang

IPC: G06F17/30

CPC classification number: G06F16/27 ,  G06F16/258

Abstract: System and method embodiments are provided for using different storage formats for a primary database and its replicas in a database managed replication (DMR) system. As such, the advantages of both formats can be combined with suitable design complexity and implementation. In an embodiment, data is arranged in a sequence of rows and stored in a first storage format at the primary database. The data arranged in the sequence of rows is also stored in a second storage format at the replica database. The sequence of rows is determined according to the first storage format or the second storage format. The first storage format is a row store (RS) and the second storage format is a column store (CS), or vice versa. In an embodiment, the sequence of rows is determined to improve compression efficiency at the CS.