Invention Grant
- Patent Title: Optimized asynchronous training of neural networks using a distributed parameter server with eager updates
-
Application No.: US15898433Application Date: 2018-02-17
-
Publication No.: US11630994B2Publication Date: 2023-04-18
- Inventor: Khaled Hamidouche , Michael W LeBeane , Walter B Benton , Michael L Chu
- Applicant: Advanced Micro Devices, Inc.
- Applicant Address: US CA Santa Clara
- Assignee: Advanced Micro Devices, Inc.
- Current Assignee: Advanced Micro Devices, Inc.
- Current Assignee Address: US CA Santa Clara
- Agency: Liang & Cheng, PC
- Main IPC: G06N3/08
- IPC: G06N3/08 ; G06F15/173 ; G06N3/084 ; G06N3/063 ; G06N3/045
Abstract:
A method of training a neural network includes, at a local computing node, receiving remote parameters from a set of one or more remote computing nodes, initiating execution of a forward pass in a local neural network in the local computing node to determine a final output based on the remote parameters, initiating execution of a backward pass in the local neural network to determine updated parameters for the local neural network, and prior to completion of the backward pass, transmitting a subset of the updated parameters to the set of remote computing nodes.
Public/Granted literature
Information query
