Invention Grant
- Patent Title: Method and device for compressing a neural network model for machine translation and storage medium
-
Application No.: US16828277Application Date: 2020-03-24
-
Publication No.: US11556761B2Publication Date: 2023-01-17
- Inventor: Xiang Li , Yuhui Sun , Jingwei Li , Jialiang Jiang
- Applicant: Beijing Xiaomi Intelligent Technology Co., Ltd.
- Applicant Address: CN Beijing
- Assignee: Beijing Xiaomi Intelligent Technology Co., Ltd.
- Current Assignee: Beijing Xiaomi Intelligent Technology Co., Ltd.
- Current Assignee Address: CN Beijing
- Agency: Finnegan, Henderson, Farabow, Garrett & Dunner, L.L.P.
- Priority: CN201911167600.6 20191125
- Main IPC: G06N3/04
- IPC: G06N3/04 ; G06F17/18 ; G06N3/08 ; G06F40/58

Abstract:
A method for compressing a neural network model includes: obtaining a first trained teacher model and a second trained teacher model based on N training samples, N being a positive integer greater than 1; for each of the N training samples, determining a first guide component of the first teacher model and a second guide component of the second teacher model respectively, determining a sub optimization target corresponding to the training sample and configured to optimize a student model according to the first guide component and the second guide component, and determining a joint optimization target based on each of the N training samples and a sub optimization target corresponding to the training sample; and training the student model based on the joint optimization target.
Public/Granted literature
- US20210158126A1 METHOD AND DEVICE FOR COMPRESSING A NEURAL NETWORK MODEL FOR MACHINE TRANSLATION AND STORAGE MEDIUM Public/Granted day:2021-05-27
Information query