Invention Grant
- Patent Title: Fault tolerance for map/reduce computing
-
Application No.: US12828247Application Date: 2010-06-30
-
Publication No.: US08381015B2Publication Date: 2013-02-19
- Inventor: David L. Kaminski
- Applicant: David L. Kaminski
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agency: Carey, Rodriquez, Greenberg & O'Keefe
- Agent Steven M. Greenberg, Esq.
- Main IPC: G06F11/00
- IPC: G06F11/00

Abstract:
Embodiments of the invention include a method for fault tolerance management of workers nodes during map/reduce computing in a computing cluster. The method includes subdividing a computational problem into a set of sub-problems, mapping a selection of the sub-problems in the set to respective nodes in the cluster, directing processing of the sub-problems in the respective nodes, and collecting results from completion of processing of the sub-problems. During a first early temporal portion of processing the computational problem, failed nodes are detected and the sub-problems currently being processed by the failed nodes are re-processed. Conversely, during a second later temporal portion of processing the computational problem, sub-problems in nodes not yet completely processed are replicated into other nodes, processing of the replicated sub-problems directed, and the results from completion of processing of sub-problems collected. Finally, duplicate results are removed and remaining results reduced into a result set for the problem.
Public/Granted literature
- US20120005522A1 FAULT TOLERANCE FOR MAP/REDUCE COMPUTING Public/Granted day:2012-01-05
Information query