公开(公告)号：WO2014164080A1

公开(公告)日：2014-10-09

申请号：PCT/US2014/020448

申请日：2014-03-05

Applicant: MICROSOFT CORPORATION

Inventor： HUANG, Jui-Ting ,  LI, Jinyu ,  YU, Dong ,  DENG, Li ,  GONG, Yifan

IPC: G10L15/16

CPC classification number: G10L15/063 ,  G06N3/0454 ,  G06N3/084 ,  G10L15/16

Abstract: Described herein are various technologies pertaining to a multilingual deep neural network (MDNN). The MDNN includes a plurality of hidden layers, wherein values for weight parameters of the plurality of hidden layers are learned during a training phase based upon training data in terms of acoustic raw features for multiple languages. The MDNN further includes softmax layers that are trained for each target language separately, making use of the hidden layer values trained jointly with multiple source languages. The MDNN is adaptable, such that a new softmax layer may be added on top of the existing hidden layers, where the new softmax layer corresponds to a new target language.

Abstract translation: 这里描述的是涉及多语言深层神经网络（MDNN）的各种技术。 MDNN包括多个隐藏层，其中基于针对多种语言的声学原始特征的训练数据，在训练阶段期间学习多个隐藏层的权重参数的值。 MDNN还包括针对每种目标语言分别训练的softmax层，利用与多种源语言联合训练的隐藏层值。 MDNN是适应性的，使得可以在现有隐藏层之上添加新的softmax层，其中新的softmax层对应于新的目标语言。

公开(公告)号：WO2016037350A1

公开(公告)日：2016-03-17

申请号：PCT/CN2014/086397

申请日：2014-09-12

Applicant: MICROSOFT CORPORATION ,  ZHAO, Rui ,  HUANG, Jui-Ting ,  LI, Jinyu ,  GONG, Yifan

Inventor： ZHAO, Rui ,  HUANG, Jui-Ting ,  LI, Jinyu ,  GONG, Yifan

IPC: G06K9/66

CPC classification number: G06N3/084 ,  G06N3/0454 ,  G06N7/005 ,  G06N99/005 ,  G09B5/00

Abstract: Systems and methods are provided for generating a DNN classifier by "learning" a "student" DNN model from a larger, more accurate "teacher" DNN model. The student DNN may be trained from unlabeled training data by passing the unlabeled training data through the teacher DNN, which may be trained from labeled data. In one embodiment, an iterative processis applied to train the student DNN by minimizing the divergence of the output distributions from the teacher and student DNN models. For each iteration until convergence, the difference in the outputs of these two DNNsis used to update the student DNN model, and outputs are determined again, using the unlabeled training data. The resulting trained student DNN model may be suitable for providing accurate signal processing applications on devices having limited computational or storage resources such as mobile or wearable devices. In an embodiment, the teacher DNN model comprises an ensemble of DNN models.

Abstract translation: 提供了通过从更大，更准确的“教师”DNN模型学习“学生”DNN模型来生成DNN分类器的系统和方法。 通过传递未标记的训练数据通过教师DNN，可以从未标记的训练数据训练学生DNN，该DNN可以从标记数据中训练。 在一个实施例中，迭代过程被应用于通过最小化来自教师和学生DNN模型的输出分布的差异来训练学生DNN。 对于每次迭代直到收敛，这两个DNNsis的输出的差异用于更新学生DNN模型，并且使用未标记的训练数据再次确定输出。 所得到的训练有素的学生DNN模型可能适合于在具有有限计算或存储资源的设备（例如移动或可穿戴设备）上提供精确的信号处理应用。 在一个实施例中，教师DNN模型包括DNN模型的集合。