公开(公告)号：WO2021062029A1

公开(公告)日：2021-04-01

申请号：PCT/US2020/052544

申请日：2020-09-24

Applicant: QUALCOMM INCORPORATED

Inventor： LU, Yadong ,  WANG, Ying ,  BLANKEVOORT, Tijmen Pieter Frederik ,  LOUIZOS, Christos ,  REISSER, Matthias ,  HOU, Jilei

IPC: G06N3/063 ,  G06N3/08 ,  G06N3/04

Abstract: A method for compressing a deep neural network includes determining a pruning ratio for a channel and a mixed-precision quantization bit-width based on an operational budget of a device implementing the deep neural network. The method further includes quantizing a weight parameter of the deep neural network and/or an activation parameter of the deep neural network based on the quantization bit-width. The method also includes pruning the channel of the deep neural network based on the pruning ratio.

公开(公告)号：EP4165529A1

公开(公告)日：2023-04-19

申请号：EP21737906.4

申请日：2021-06-11

Applicant: QUALCOMM INCORPORATED

Inventor： HOSSEINI, Hossein ,  LOUIZOS, Christos ,  SORIAGA, Joseph Binamira

IPC: G06F21/31 ,  G06F21/32 ,  G06N3/08

公开(公告)号：WO2021158830A1

公开(公告)日：2021-08-12

申请号：PCT/US2021/016680

申请日：2021-02-04

Applicant: QUALCOMM INCORPORATED

Inventor： AMJAD, Rana Ali ,  NAGEL, Markus ,  BLANKEVOORT, Tijmen Pieter Frederik ,  VAN BAALEN, Marinus Willem ,  LOUIZOS, Christos

IPC: G06N3/063 ,  G06N3/08 ,  G06N3/04

Abstract: A method for quantizing a pre-trained neural network includes computing a loss on a training set of candidate weights of the neural network. A rounding parameter is assigned to each candidate weight. The rounding parameter is a binary random value or a multinomial value. A quantized weight value is computed based on the loss and the rounding parameter.

公开(公告)号：WO2019222543A1

公开(公告)日：2019-11-21

申请号：PCT/US2019/032732

申请日：2019-05-16

Applicant: QUALCOMM INCORPORATED

Inventor： LOUIZOS, Christos ,  REISSER, Matthias ,  BLANKEVOORT, Tijmen Pieter Frederik ,  WELLING, Max

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08

Abstract: A method for quantizing a neural network includes modeling noise of parameters of the neural network. The method also includes assigning grid values to each realization of the parameters according to a concrete distribution that depends on a local fixed-point quantization grid and the modeled noise and. The method further includes computing a fixed-point value representing parameters of a hard fixed-point quantized neural network.

公开(公告)号：WO2022251885A1

公开(公告)日：2022-12-01

申请号：PCT/US2022/072659

申请日：2022-05-31

Applicant: QUALCOMM INCORPORATED

Inventor： REISSER, Matthias ,  TRIASTCYN, Aleksei ,  LOUIZOS, Christos

IPC: H03M7/30 ,  G06N3/08 ,  G06N20/20

Abstract: Certain aspects of the present disclosure provide techniques for performing federated learning, including receiving a global model from a federated learning server; determining an updated model based on the global model and local data; and sending the updated model to the federated learning server using relative entropy coding.

公开(公告)号：WO2022067355A1

公开(公告)日：2022-03-31

申请号：PCT/US2021/071633

申请日：2021-09-28

Applicant: QUALCOMM INCORPORATED

Inventor： LOUIZOS, Christos ,  HOSSEINI, Hossein ,  REISSER, Matthias ,  WELLING, Max ,  SORIAGA, Joseph Binamira

IPC: G06N3/04 ,  G06N3/08 ,  G06N7/00 ,  G06N20/00

Abstract: Aspects described herein provide techniques for performing federated learning of a machine learning model, comprising: for each respective client of a plurality of clients and for each training round in a plurality of training rounds: generating a subset of model elements for the respective client based on sampling a gate probability distribution for each model element of a set of model elements for a global machine learning model; transmitting to the respective client: the subset of model elements; and a set of gate probabilities based on the sampling, wherein each gate probability of the set of gate probabilities is associated with one model element of the subset of model elements; receiving from each respective client of the plurality of clients a respective set of model updates; and updating the global machine learning model based on the respective set of model updates from each respective client of the plurality of clients.

公开(公告)号：WO2021222656A1

公开(公告)日：2021-11-04

申请号：PCT/US2021/030009

申请日：2021-04-29

Applicant: QUALCOMM INCORPORATED

Inventor： VAN BAALEN, Marinus Willem ,  LOUIZOS, Christos ,  NAGEL, Markus ,  BLANKEVOORT, Tijmen Pieter Frederik ,  AMJAD, Rana Ali

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

Abstract: Various embodiments include methods and devices for joint mixed-precision quantization and structured pruning. Embodiments may include determining whether a plurality of gates of quantization and pruning gates are selected for combination, and in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor in which the next bit-width increases for each successive iteration, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor in which the next quantized tensor has the next bit-width, and in which the next quantized tensor is the quantized tensor for a successive iteration.

公开(公告)号：EP4320556A1

公开(公告)日：2024-02-14

申请号：EP22719189.7

申请日：2022-04-04

Applicant: QUALCOMM INCORPORATED

Inventor： GUO, Yunhui ,  HOSSEINI, Hossein ,  LOUIZOS, Christos ,  SORIAGA, Joseph Binamira

IPC: G06N3/04 ,  G06N3/08

公开(公告)号：EP4217931A1

公开(公告)日：2023-08-02

申请号：EP21798250.3

申请日：2021-09-28

Applicant: QUALCOMM INCORPORATED

Inventor： LOUIZOS, Christos ,  HOSSEINI, Hossein ,  REISSER, Matthias ,  WELLING, Max ,  SORIAGA, Joseph Binamira

IPC: G06N3/04 ,  G06N3/08 ,  G06N7/00 ,  G06N20/00

公开(公告)号：EP4526811A1

公开(公告)日：2025-03-26

申请号：EP23732787.9

申请日：2023-05-17

Applicant: QUALCOMM INCORPORATED

Inventor： LOUIZOS, Christos ,  TRIASTCYN, Aleksei

IPC: G06N3/098 ,  G06N20/20