Invention Grant
- Patent Title: System for efficient large-scale data distribution in distributed and parallel processing environment
-
Application No.: US16731095Application Date: 2019-12-31
-
Publication No.: US11436065B2Publication Date: 2022-09-06
- Inventor: Xiaowen Chu , Shaohuai Shi , Kaiyong Zhao
- Applicant: Hong Kong Baptist University
- Applicant Address: HK Hong Kong
- Assignee: Hong Kong Baptist University
- Current Assignee: Hong Kong Baptist University
- Current Assignee Address: HK Hong Kong
- Agency: Idea Intellectual Limited
- Agent Margaret A. Burke; Sam T. Yip
- Main IPC: G06F9/54
- IPC: G06F9/54 ; G06N3/08 ; G06N3/04 ; H04L69/04 ; H04L9/40
Abstract:
The present invention relates to a system for efficient large-scale data distribution in a distributed and parallel processing environment. In particular, the present invention relates to global Top-k sparsification for low bandwidth networks. The present invention verifies that gTop-k S-SGD has nearly consistent convergence performance with S-SGD and evaluates the training efficiency of gTop-k on a cluster with 32 GPU machines which are inter-connected with 1 Gbps Ethernet. The experimental results show that the present invention achieves up to 2.7-12× higher scaling efficiency than S-SGD with dense gradients, and 1.1-1.7× improvement than the existing Top-k S-SGD.
Public/Granted literature
- US20210200610A1 SYSTEM FOR EFFICIENT LARGE-SCALE DATA DISTRIBUTION IN DISTRIBUTED AND PARALLEL PROCESSING ENVIRONMENT Public/Granted day:2021-07-01
Information query
