Invention Grant
- Patent Title: Lazy data loading for improving memory cache hit ratio in DAG-based computational system
-
Application No.: US15969375Application Date: 2018-05-02
-
Publication No.: US10671436B2Publication Date: 2020-06-02
- Inventor: Tatsuhiro Chiba , Takeshi Yoshimura
- Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
- Applicant Address: US NY Armonk
- Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
- Current Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
- Current Assignee Address: US NY Armonk
- Agency: Tutunjian & Bitetto, P.C.
- Agent Vazken Alexanian
- Main IPC: G06F9/50
- IPC: G06F9/50 ; G06F9/48 ; G06F12/121 ; G06F12/0895 ; G06F16/22 ; G06F16/901

Abstract:
A method is provided for improving a hit ratio of a buffer cache in a system in which vertices of a DAG have tasks that generate intermediate data stored in the buffer cache. The method tracks (i) a buffer cache usage by vertices that have finished running and (ii) a current available buffer cache space. Responsive to a new task being runnable and having dependent parent vertices, the method estimates a total buffer cache usage of current running vertices based on a partial result of the current running vertices. Responsive to the estimate exceeding current available buffer cache space, the method (i) selects a vertex having a most amount of intermediate data stored in the buffer cache for its dependent parent vertices, and (ii) increases a priority of the tasks in the selected vertex to obtain prioritized tasks. The method executes the prioritized tasks earlier than other remaining runnable tasks.
Public/Granted literature
- US20190340016A1 LAZY DATA LOADING FOR IMPROVING MEMORY CACHE HIT RATIO IN DAG-BASED COMPUTATIONAL SYSTEM Public/Granted day:2019-11-07
Information query