公开(公告)号：US20220300310A1

公开(公告)日：2022-09-22

申请号：US17834256

申请日：2022-06-07

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/455 ,  G06F21/62 ,  G06F9/38 ,  G06F8/41 ,  G06F9/48 ,  G06F8/30 ,  G06F8/34 ,  G06F9/44 ,  G06F9/445 ,  G06F21/57

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20210397534A1

公开(公告)日：2021-12-23

申请号：US17465397

申请日：2021-09-02

Applicant: Google LLC

Inventor： Mariana Sharp ,  Craig D. Chambers ,  Jelena Pjesivac-Grbovic ,  Kym Hines ,  Sunghwan Ihm ,  Danijela Mijailovic ,  Ben Walters

IPC: G06F11/34 ,  G06F11/30 ,  G06F9/46

Abstract: Aspects of the disclosure relate to evaluating the rollout of a change to a data processing framework (DPF). A first set of jobs is selected out of a plurality of jobs. The first set of jobs is processed by the DPF with the proposed change enabled and the other jobs of the plurality of jobs by the DPF without the change enabled. A performance cost value is determined for each of the jobs in the first set of jobs processed by the DPF with the proposed change enabled and for each of the other jobs of the plurality of jobs processed by the DPF with the proposed change disabled. Based on the performance cost values of the jobs in the first set of jobs and the performance cost values of the other jobs of the plurality of jobs, a determination is made whether the proposed change is beneficial to the DPF.

公开(公告)号：US10795705B2

公开(公告)日：2020-10-06

申请号：US16449987

申请日：2019-06-24

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/44 ,  G06F9/46 ,  G06F9/48 ,  G06F9/50 ,  H04L29/08 ,  G06F9/455 ,  G06F21/62 ,  G06F9/38 ,  G06F8/41 ,  G06F8/30 ,  G06F8/34 ,  G06F9/445 ,  G06F21/57 ,  G06F16/2453 ,  G06F9/448 ,  G06F9/30

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20190317782A1

公开(公告)日：2019-10-17

申请号：US16449987

申请日：2019-06-24

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/455 ,  G06F9/38 ,  G06F8/30 ,  G06F8/34 ,  G06F9/44 ,  G06F21/62 ,  G06F8/41 ,  G06F9/48 ,  G06F21/57 ,  G06F9/445

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US10338942B2

公开(公告)日：2019-07-02

申请号：US16175925

申请日：2018-10-31

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F17/30 ,  G06F9/44 ,  G06F9/48 ,  G06F9/46 ,  G06F9/455 ,  G06F9/38 ,  G06F21/62 ,  G06F8/41 ,  G06F21/57 ,  G06F9/445 ,  G06F8/34 ,  G06F8/30 ,  G06F9/448 ,  G06F9/30 ,  G06F16/2453

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20190065224A1

公开(公告)日：2019-02-28

申请号：US16175925

申请日：2018-10-31

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/455 ,  G06F21/62 ,  G06F8/30 ,  G06F8/34 ,  G06F8/41 ,  G06F9/38 ,  G06F21/57 ,  G06F9/48 ,  G06F9/445 ,  G06F9/44 ,  G06F17/30 ,  G06F9/30 ,  G06F9/448

CPC classification number: G06F9/45504 ,  G06F8/314 ,  G06F8/34 ,  G06F8/433 ,  G06F9/30 ,  G06F9/38 ,  G06F9/3851 ,  G06F9/3885 ,  G06F9/44 ,  G06F9/445 ,  G06F9/4494 ,  G06F9/45533 ,  G06F9/4843 ,  G06F16/24532 ,  G06F16/24547 ,  G06F21/577 ,  G06F21/62 ,  G06F21/6218 ,  G06F2221/034

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.