Invention Grant
- Patent Title: Neural network training with decreased memory consumption and processor utilization
-
Application No.: US16550229Application Date: 2019-08-24
-
Publication No.: US11526761B2Publication Date: 2022-12-13
- Inventor: Taesik Na , Daniel Lo , Haishan Zhu , Eric Sen Chung
- Applicant: Microsoft Technology Licensing LLC
- Applicant Address: US WA Redmond
- Assignee: Microsoft Technology Licensing LLC
- Current Assignee: Microsoft Technology Licensing LLC
- Current Assignee Address: US WA Redmond
- Agency: Alleman Hall Creasman & Tuttle LLP
- Main IPC: G06N3/08
- IPC: G06N3/08

Abstract:
Bounding box quantization can reduce the quantity of bits utilized to express numerical values prior to the multiplication of matrices comprised of such numerical values, thereby reducing both memory consumption and processor utilization. Stochastic rounding can provide sufficient precision to enable the storage of weight values in reduced-precision formats without having to separately store weight values in a full-precision format. Alternatively, other rounding mechanisms, such as round to nearest, can be utilized to exchange weight values in reduced-precision formats, while also storing weight values in full-precision formats for subsequent updating. To facilitate conversion, reduced-precision formats such as brain floating-point format can be utilized.
Public/Granted literature
- US20210056423A1 Neural Network Training With Decreased Memory Consumption And Processor Utilization Public/Granted day:2021-02-25
Information query