Invention Grant
- Patent Title: Streamlining data processing optimizations for machine learning workloads
-
Application No.: US16890091Application Date: 2020-06-02
-
Publication No.: US11574249B2Publication Date: 2023-02-07
- Inventor: Qi Zhang , Petr Novotny , Hong Min , Ravi Nair , Shyam Ramji , Lei Yu , Takuya Nakaike , Motohiro Kawahito
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agency: Patterson + Sheridan, LLP
- Main IPC: G06N20/00
- IPC: G06N20/00 ; G06F16/25

Abstract:
Techniques for refinement of data pipelines are provided. An original file of serialized objects is received, and an original pipeline comprising a plurality of transformations is identified based on the original file. A first computing cost is determined for a first transformation of the plurality of transformations. The first transformation is modified using a predefined optimization, and a second cost of the modified first transformation is determined. Upon determining that the second cost is lower than the first cost, the first transformation is replaced, in the original pipeline, with the optimized first transformation.
Public/Granted literature
- US20210374602A1 STREAMLINING DATA PROCESSING OPTIMIZATIONS FOR MACHINE LEARNING WORKLOADS Public/Granted day:2021-12-02
Information query