公开(公告)号：US20220366263A1

公开(公告)日：2022-11-17

申请号：US17313655

申请日：2021-05-06

Applicant: Waymo LLC

Inventor： Ming Ji ,  Edward Stephen Walker, JR. ,  Yang Song ,  Zijian Guo ,  Congcong Li

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a student machine learning model using a teacher machine learning model that has a pre-trained feature extractor. In one aspect, a method includes obtaining data specifying the teacher machine learning model that is configured to perform a machine learning task; obtaining first training data; training the teacher machine learning model on the first training data to obtain a trained teacher machine learning model; generating second, automatically labeled training data by using the trained teacher machine learning model to process unlabeled training data; and training a student machine learning model to perform the machine learning task using at least the second, automatically labeled training data, wherein the student machine learning model does not include the pre-trained feature extractor and instead includes a different feature extractor having fewer parameters than the pre-trained feature extractor.

公开(公告)号：US20210334651A1

公开(公告)日：2021-10-28

申请号：US17194115

申请日：2021-03-05

Applicant: Waymo LLC

Inventor： Zhaoqi Leng ,  Ekin Dogus Cubuk ,  Barret Zoph ,  Jiquan Ngiam ,  Congcong Li ,  Jonathon Shlens ,  Shuyang Cheng

IPC: G06N3/08 ,  G06F17/18 ,  G06K9/62 ,  G01S17/894

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model to perform a machine learning task by processing input data to the model. For example, the input data can include image, video, or point cloud data, and the task can be a perception task such as classification or detection task. In one aspect, the method includes receiving training data including a plurality of training inputs; receiving a plurality of data augmentation policy parameters that define different transformation operations for transforming training inputs before the training inputs are used to train the machine learning model; maintaining a plurality of candidate machine learning models; for each of the plurality of candidate machine learning models: repeatedly determining an augmented batch of training data; training the candidate machine learning model using the augmented batch of the training data; and updating the maintained data.

公开(公告)号：US11093819B1

公开(公告)日：2021-08-17

申请号：US15381389

申请日：2016-12-16

Applicant: Waymo LLC

Inventor： Congcong Li ,  Ury Zhilinsky ,  Yun Jiang ,  Zhaoyin Jia

IPC: G06N3/04 ,  G06N3/08

Abstract: Disclosed herein are neural networks for generating target classifications for an object from a set of input sequences. Each input sequence includes a respective input at each of multiple time steps, and each input sequence corresponds to a different sensing subsystem of multiple sensing subsystems. For each time step in the multiple time steps and for each input sequence in the set of input sequences, a respective feature representation is generated for the input sequence by processing the respective input from the input sequence at the time step using a respective encoder recurrent neural network (RNN) subsystem for the sensing subsystem that corresponds to the input sequence. For each time step in at least a subset of the multiple time steps, the respective feature representations are processed using a classification neural network subsystem to select a respective target classification for the object at the time step.

公开(公告)号：US20210192238A1

公开(公告)日：2021-06-24

申请号：US17123185

申请日：2020-12-16

Applicant: WAYMO LLC

Inventor： Victoria Dean ,  Abhijit S. Ogale ,  Henrik Kretzschmar ,  David Harrison Silver ,  Carl Kershaw ,  Pankaj Chaudhari ,  Chen Wu ,  Congcong Li

IPC: G06K9/00 ,  G05D1/00 ,  G06N3/08 ,  G06T11/20 ,  G06K9/34 ,  G06F40/30

Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.

公开(公告)号：US20210191395A1

公开(公告)日：2021-06-24

申请号：US16723787

申请日：2019-12-20

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Junhua Mao ,  Yi Shen ,  Congcong Li ,  Chen Sun

IPC: G05D1/00 ,  G06K9/00 ,  G06K9/62 ,  G05D1/02 ,  B60W60/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.

公开(公告)号：US10366502B1

公开(公告)日：2019-07-30

申请号：US15374884

申请日：2016-12-09

Applicant: Waymo LLC

Inventor： Congcong Li

IPC: G06K9/00 ,  G05D1/00 ,  G06T7/50 ,  G06N3/04 ,  G06N3/08 ,  G06T7/73 ,  B60R1/00 ,  G05D1/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating vehicle heading predictions from point cloud data using a neural network. One of the methods includes receiving a plurality of different projections of point cloud data, wherein the point cloud data represents different sensor measurements of electromagnetic radiation reflected off a vehicle. Each of the plurality of projections of point cloud data is provided as input to a neural network subsystem trained to receive projections of point cloud data for a vehicle and to generate one or more vehicle heading classifications as an output. At the output of the neural network subsystem, one or more vehicle heading predictions is received.

公开(公告)号：US20240149906A1

公开(公告)日：2024-05-09

申请号：US17387852

申请日：2021-07-28

Applicant: Waymo LLC

Inventor： Hang Zhao ,  Jiyang Gao ,  Chen Sun ,  Yi Shen ,  Yuning Chai ,  Cordelia Luise Schmid ,  Congcong Li ,  Benjamin Sapp ,  Dragomir Anguelov ,  Tian Lan ,  Yue Shen

IPC: B60W60/00 ,  G06N3/02

CPC classification number: B60W60/001 ,  G06N3/02 ,  B60W2420/42 ,  B60W2554/4049

Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for predicting future trajectories for an agent in an environment. A system obtains scene context data characterizing the environment. The scene context data includes data that characterizes a trajectory of an agent in a vicinity of a vehicle in an. environment up to a current time point. The system identifies a plurality of initial target locations in the environment. The system further generates, for each of a plurality of target locations that each corresponds to one of the initial target locations, a respective predicted likelihood score that represents a likelihood that the target location will be an intended final location for a future trajectory of the agent starting from the current time point. For each target location in a first subset of the target locations, the system generates a predicted future trajectory for the agent that is a prediction of the future trajectory of the agent given that the target location is the intended final location for the future trajectory. The system further selects, as likely future trajectories of the agent starting from the current time point, one or more of the predicted future trajectories.

公开(公告)号：US11782158B2

公开(公告)日：2023-10-10

申请号：US16229370

申请日：2018-12-21

Applicant: Waymo LLC

Inventor： David Lee ,  Xiaohan Jin ,  Congcong Li ,  Nichola Abdo

IPC: G01S17/58 ,  G01S17/06 ,  G01S17/89 ,  G06N3/08 ,  G06V10/82 ,  G06N7/01 ,  G06T7/20 ,  G06V20/58 ,  B60W30/095

CPC classification number: G01S17/58 ,  G01S17/06 ,  G01S17/89 ,  G06N3/08 ,  G06N7/01 ,  G06V10/82 ,  B60W30/0956 ,  G06T7/20 ,  G06T2207/10028 ,  G06T2207/30241 ,  G06T2207/30252 ,  G06V20/58

Abstract: Systems, methods, devices, and techniques for generating object-heading estimations. In one example, methods include actions of receiving sensor data representing measurements of an object that was detected within a proximity of a vehicle; processing the sensor data with one or more preliminary heading estimation subsystems to respectively generate one or more preliminary heading estimations for the object; processing two or more inputs with a second heading estimation subsystem to generate a refined heading estimation for the object, the two or more inputs including the one or more preliminary heading estimations for the object; and providing the refined heading estimation for the object to an external processing system.

公开(公告)号：US11657291B2

公开(公告)日：2023-05-23

申请号：US17063553

申请日：2020-10-05

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Zijian Guo ,  Congcong Li

IPC: G06V20/58 ,  G06N3/08 ,  G06N3/04 ,  G06V10/75

CPC classification number: G06V20/58 ,  G06N3/0454 ,  G06N3/08 ,  G06V10/757

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a spatio-temporal embedding of a sequence of point clouds. One of the methods includes obtaining a temporal sequence comprising a respective point cloud input corresponding to each of a plurality of time points, each point cloud input comprising point cloud data generated from sensor data captured by one or more sensors of a vehicle at the respective time point; processing each point cloud input using a first neural network to generate a respective spatial embedding that characterizes the point cloud input; and processing the spatial embeddings of the point cloud inputs using a second neural network to generate a spatio-temporal embedding that characterizes the point cloud inputs in the temporal sequence.

公开(公告)号：US11514310B2

公开(公告)日：2022-11-29

申请号：US16231297

申请日：2018-12-21

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Lo Po Tsui ,  Congcong Li ,  Edward Stephen Walker, Jr.

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a classifier to detect open vehicle doors. One of the methods includes obtaining a plurality of initial training examples, each initial training example comprising (i) a sensor sample from a collection of sensor samples and (ii) data classifying the sensor sample as characterizing a vehicle that has an open door; generating a plurality of additional training examples, comprising, for each initial training example: identifying, from the collection of sensor samples, one or more additional sensor samples that were captured less than a threshold amount of time before the sensor sample in the initial training example was captured; and training the machine learning classifier on first training data that includes the initial training examples and the additional training examples to generate updated weights for the machine learning classifier.