公开(公告)号：US12073823B2

公开(公告)日：2024-08-27

申请号：US18506540

申请日：2023-11-10

Applicant: Google LLC

Inventor： Georg Heigold ,  Erik Mcdermott ,  Vincent O. Vanhoucke ,  Andrew W. Senior ,  Michiel A. U. Bacchiani

IPC: G10L15/06 ,  G06N3/045 ,  G10L15/16 ,  G10L15/183

CPC classification number: G10L15/063 ,  G06N3/045 ,  G10L15/16 ,  G10L15/183

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining, by a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, by the first sequence-training speech model, one or more first neural network parameters; determining, by the first sequence-training speech model, one or more optimized first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; obtaining, by a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances; obtaining one or more second neural network parameters; and determining, by the second sequence-training speech model, one or more optimized second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters.

公开(公告)号：US20240062426A1

公开(公告)日：2024-02-22

申请号：US18500034

申请日：2023-11-01

Applicant: Google LLC

Inventor： Neil Matthew Tinmouth Houlsby ,  Sylvain Gelly ,  Jakob D. Uszkoreit ,  Xiaohua Zhai ,  Georg Heigold ,  Lucas Klaus Beyer ,  Alexander Kolesnikov ,  Matthias Johannes Lorenz Minderer ,  Dirk Weissenborn ,  Mostafa Dehghani ,  Alexey Dosovitskiy ,  Thomas Unterthiner

IPC: G06T7/00 ,  G06F18/24 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06T7/97 ,  G06F18/24 ,  G06N3/045 ,  G06N3/08 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using self-attention based neural networks. One of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more self-attention neural network layers.

公开(公告)号：US20220383628A1

公开(公告)日：2022-12-01

申请号：US17726374

申请日：2022-04-21

Applicant: Google LLC

Inventor： Thomas Kipf ,  Gamaleldin Elsayed ,  Aravindh Mahendran ,  Austin Charles Stone ,  Sara Sabour Rouh Aghdam ,  Georg Heigold ,  Rico Jonschkowski ,  Alexey Dosovitskiy ,  Klaus Greff

IPC: G06V10/82 ,  G06V10/40 ,  G06V10/774

Abstract: A method includes obtaining first feature vectors and second feature vectors representing contents of a first and second image frame, respectively, of an input video. The method may also include generating, based on the first feature vectors, first slot vectors, where each slot vector represents attributes of a corresponding entity as represented in the first image frame, and generating, based on the first slot vectors, predicted slot vectors including a corresponding predicted slot vector that represents a transition of the attributes of the corresponding entity from the first to the second image frame. The method may additionally include generating, based on the predicted slot vectors and the second feature vectors, second slot vectors including a corresponding slot vector that represents the attributes of the corresponding entity as represented in the second image frame, and determining an output based on the predicted slot vectors or the second slot vectors.

公开(公告)号：US20220108686A1

公开(公告)日：2022-04-07

申请号：US17644362

申请日：2021-12-15

Applicant: Google LLC

Inventor： Georg Heigold ,  Erik McDermott ,  Vincent O. VanHoucke ,  Andrew W. Senior ,  Michiel A.U. Bacchiani

IPC: G10L15/06 ,  G10L15/16 ,  G10L15/183 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining, by a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, by the first sequence-training speech model, one or more first neural network parameters; determining, by the first sequence-training speech model, one or more optimized first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; obtaining, by a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances; obtaining one or more second neural network parameters; and determining, by the second sequence-training speech model, one or more optimized second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters.

公开(公告)号：US11227582B2

公开(公告)日：2022-01-18

申请号：US17143140

申请日：2021-01-06

Applicant: Google LLC

Inventor： Georg Heigold ,  Erik Mcdermott ,  Vincent O. Vanhoucke ,  Andrew W. Senior ,  Michiel A. U. Bacchiani

IPC: G10L15/06 ,  G10L15/16 ,  G10L15/183 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining, by a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, by the first sequence-training speech model, one or more first neural network parameters; determining, by the first sequence-training speech model, one or more optimized first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; obtaining, by a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances; obtaining one or more second neural network parameters; and determining, by the second sequence-training speech model, one or more optimized second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters.

公开(公告)号：US20210383199A1

公开(公告)日：2021-12-09

申请号：US16927018

申请日：2020-07-13

Applicant: Google LLC

Inventor： Dirk Weissenborn ,  Jakob Uszkoreit ,  Thomas Unterthiner ,  Aravindh Mahendran ,  Francesco Locatello ,  Thomas Kipf ,  Georg Heigold ,  Alexey Dosovitskiy

IPC: G06N3/063 ,  G06N3/08 ,  G06F17/16

Abstract: A method involves receiving a perceptual representation including a plurality of feature vectors, and initializing a plurality of slot vectors represented by a neural network memory unit. Each respective slot vector is configured to represent a corresponding entity in the perceptual representation. The method also involves determining an attention matrix based on a product of the plurality of feature vectors transformed by a key function and the plurality of slot vectors transformed by a query function. Each respective value of a plurality of values along each respective dimension of the attention matrix is normalized with respect to the plurality of values. The method additionally involves determining an update matrix based on the plurality of feature vectors transformed by a value function and the attention matrix, and updating the plurality of slot vectors based on the update matrix by way of the neural network memory unit.

公开(公告)号：US20180315430A1

公开(公告)日：2018-11-01

申请号：US15966667

申请日：2018-04-30

Applicant: Google LLC

Inventor： Georg Heigold ,  Samuel Bengio ,  Ignacio Lopez Moreno

IPC: G10L17/18 ,  G10L17/02 ,  G10L17/04

Abstract: This document generally describes systems, methods, devices, and other techniques related to speaker verification, including (i) training a neural network for a speaker verification model, (ii) enrolling users at a client device, and (iii) verifying identities of users based on characteristics of the users' voices. Some implementations include a computer-implemented method. The method can include receiving, at a computing device, data that characterizes an utterance of a user of the computing device. A speaker representation can be generated, at the computing device, for the utterance using a neural network on the computing device. The neural network can be trained based on a plurality of training samples that each: (i) include data that characterizes a first utterance and data that characterizes one or more second utterances, and (ii) are labeled as a matching speakers sample or a non-matching speakers sample.

公开(公告)号：US20240428586A1

公开(公告)日：2024-12-26

申请号：US18827088

申请日：2024-09-06

Applicant: Google LLC

Inventor： Anurag Arnab ,  Mostafa Dehghani ,  Georg Heigold ,  Chen Sun ,  Mario Lucic ,  Cordelia Luise Schmid

IPC: G06V20/40 ,  G06N20/00

Abstract: A computer-implemented method for classifying video data with improved accuracy includes obtaining, by a computing system comprising one or more computing devices, video data comprising a plurality of video frames; extracting, by the computing system, a plurality of spatiotemporal representations from the video data, the plurality of spatiotemporal representations comprising a representation of spatiotemporal information in the video data; providing, by the computing system, the plurality of spatiotemporal representations as input to a video understanding model, the video understanding model comprising a video transformer encoder model; and receiving, by the computing system, a classification output from the video understanding model.

公开(公告)号：US11854534B1

公开(公告)日：2023-12-26

申请号：US18069035

申请日：2022-12-20

Applicant: Google LLC

Inventor： Georg Heigold ,  Erik Mcdermott ,  Vincent O. Vanhoucke ,  Andrew W. Senior ,  Michiel A. U. Bacchiani

IPC: G10L15/06 ,  G10L15/183 ,  G06N3/045

CPC classification number: G10L15/063 ,  G06N3/045 ,  G10L15/06 ,  G10L15/183

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining, by a first sequence-training speech model, a first batch of training frames that represent speech features of first training utterances; obtaining, by the first sequence-training speech model, one or more first neural network parameters; determining, by the first sequence-training speech model, one or more optimized first neural network parameters based on (i) the first batch of training frames and (ii) the one or more first neural network parameters; obtaining, by a second sequence-training speech model, a second batch of training frames that represent speech features of second training utterances; obtaining one or more second neural network parameters; and determining, by the second sequence-training speech model, one or more optimized second neural network parameters based on (i) the second batch of training frames and (ii) the one or more second neural network parameters.

公开(公告)号：US11107478B2

公开(公告)日：2021-08-31

申请号：US16752007

申请日：2020-01-24

Applicant: Google LLC

Inventor： Georg Heigold ,  Samuel Bengio ,  Ignacio Lopez Moreno

IPC: G10L17/18 ,  G10L17/04 ,  G10L17/02

Abstract: This document generally describes systems, methods, devices, and other techniques related to speaker verification, including (i) training a neural network for a speaker verification model, (ii) enrolling users at a client device, and (iii) verifying identities of users based on characteristics of the users' voices. Some implementations include a computer-implemented method. The method can include receiving, at a computing device, data that characterizes an utterance of a user of the computing device. A speaker representation can be generated, at the computing device, for the utterance using a neural network on the computing device. The neural network can be trained based on a plurality of training samples that each: (i) include data that characterizes a first utterance and data that characterizes one or more second utterances, and (ii) are labeled as a matching speakers sample or a non-matching speakers sample.