Invention Grant
- Patent Title: Data processing method and apparatus
-
Application No.: US16259138Application Date: 2019-01-28
-
Publication No.: US11640257B2Publication Date: 2023-05-02
- Inventor: Cheng He , Qun Huang , Pak-Ching Lee
- Applicant: HUAWEI TECHNOLOGIES CO., LTD.
- Applicant Address: CN Guangdong
- Assignee: HUAWEI TECHNOLOGIES CO., LTD.
- Current Assignee: HUAWEI TECHNOLOGIES CO., LTD.
- Current Assignee Address: CN Guangdong
- Priority: CN201610616360.3 20160728
- Main IPC: G06F3/06
- IPC: G06F3/06 ; G06F11/14

Abstract:
In a data processing method, a worker node in a distributed data processing system receives first data from an upstream worker node. The first data has been stored in a buffer of the upstream worker node. The worker node sends a first portion of the first data to a persistent storage device of the distributed data processing system for persistent backup, and performs computational processing on the first data to generate second data. Prior to completing performing computational processing on the first data, the worker node sends acknowledgement information to the upstream worker node to instruct the upstream node to delete the first data from the buffer of the upstream worker node. The worker node then sends the second data to a downstream worker node in the distributed data processing system for further processing by the downstream worker node.
Public/Granted literature
- US20190171367A1 DATA PROCESSING METHOD AND APPARATUS Public/Granted day:2019-06-06
Information query