Invention Grant
- Patent Title: Determining a quantity of remote shared partitions based on mapper and reducer nodes
-
Application No.: US16006503Application Date: 2018-06-12
-
Publication No.: US10915365B2Publication Date: 2021-02-09
- Inventor: Jiyuan Tang , Wei Wang , Yi Cai
- Applicant: Huawei Technologies Co., Ltd.
- Applicant Address: CN Shenzhen
- Assignee: Huawei Technologies Co., Ltd.
- Current Assignee: Huawei Technologies Co., Ltd.
- Current Assignee Address: CN Shenzhen
- Agency: Conley Rose, P.C.
- Main IPC: G06F9/50
- IPC: G06F9/50 ; G06F15/167 ; G06F3/06 ; G06F16/182

Abstract:
A mapper node and a reducer node respectively run on different central processing units (CPUs) in a CPU pool, and a remote shared partition shared by the mapper node and the reducer node is delimited in the storage pool. The mapper node executes a map task to obtain a data segment, and stores the data segment into a remote shared partition. The reducer node directly obtains a to-be-processed data segment from the remote shared partition, and executes a reduce task on the to-be-processed data segment.
Public/Granted literature
- US20180293108A1 Data Processing Method and Apparatus, and System Public/Granted day:2018-10-11
Information query