公开(公告)号：DE69030228T2

公开(公告)日：1997-09-18

申请号：DE69030228

申请日：1990-10-04

Applicant: IBM

Inventor： DIAS DANIEL MANUEL ,  WOLF JOEL LEONARD ,  SHI-LUNG YU PHILIP

IPC: G06F15/16 ,  G06F9/50 ,  G06F12/00 ,  G06F15/177 ,  G06F17/30 ,  G06F17/40

Abstract: A technique for performing joins in parallel on a multiple processor database system effectively deals with data skew. The join operation is performed in three stages with an optional fourth stage. The first stage is a preparatory stage, the detail of which depends on the underlying join algorithm used. This preparatory stage provides pre-processing the results of which are used in the following stage as the basis for defining subtasks for the final join operation. The data provided in the first stage is used in the second stage to both define subtasks and to optimally allocate these subtasks to different processors in such a manner that the processors are close to equally loaded in the final join operation, even in the presence of data skew. This second stage is an assignment stage the details of which depend on the underlying join algorithm. Once the second stage has completed its processing of the subtasks, the subtasks are shipped to their assigned processors for processing and the final join of the two relations in the third stage. The method used in the final join operation depends on the underlying join algorithm used. Optionally, during the actual join as performed in the third stage, there could be a dynamic re-assignment of the subtasks should the join operation become unbalanced.

公开(公告)号：DE69030228D1

公开(公告)日：1997-04-24

申请号：DE69030228

申请日：1990-10-04

Applicant: IBM

Inventor： DIAS DANIEL MANUEL ,  WOLF JOEL LEONARD ,  SHI-LUNG YU PHILIP

IPC: G06F15/16 ,  G06F9/50 ,  G06F12/00 ,  G06F15/177 ,  G06F17/30 ,  G06F17/40

Abstract: A technique for performing joins in parallel on a multiple processor database system effectively deals with data skew. The join operation is performed in three stages with an optional fourth stage. The first stage is a preparatory stage, the detail of which depends on the underlying join algorithm used. This preparatory stage provides pre-processing the results of which are used in the following stage as the basis for defining subtasks for the final join operation. The data provided in the first stage is used in the second stage to both define subtasks and to optimally allocate these subtasks to different processors in such a manner that the processors are close to equally loaded in the final join operation, even in the presence of data skew. This second stage is an assignment stage the details of which depend on the underlying join algorithm. Once the second stage has completed its processing of the subtasks, the subtasks are shipped to their assigned processors for processing and the final join of the two relations in the third stage. The method used in the final join operation depends on the underlying join algorithm used. Optionally, during the actual join as performed in the third stage, there could be a dynamic re-assignment of the subtasks should the join operation become unbalanced.