公开(公告)号：US20220374650A1

公开(公告)日：2022-11-24

申请号：US17836287

申请日：2022-06-09

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Congcong Li ,  Yang Song

IPC: G06K9/62 ,  G05D1/02 ,  G06N3/04 ,  G06N3/08 ,  G06V20/56

Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.

公开(公告)号：US11480963B2

公开(公告)日：2022-10-25

申请号：US16723787

申请日：2019-12-20

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Junhua Mao ,  Yi Shen ,  Congcong Li ,  Chen Sun

IPC: G01C22/00 ,  G05D1/00 ,  G06K9/62 ,  G05D1/02 ,  B60W60/00 ,  G06V20/58 ,  G06V20/56

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.

公开(公告)号：US20210284184A1

公开(公告)日：2021-09-16

申请号：US17194072

申请日：2021-03-05

Applicant: Waymo LLC

Inventor： Yang Song ,  Shuyang Cheng ,  Zijian Guo ,  Congcong Li ,  Chunyan Bai

IPC: B60W50/06 ,  G06N3/04 ,  G06N3/08 ,  B60W60/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a point cloud augmentation policy and training a machine learning model using the point cloud augmentation policy to perform a perception task such as object detection or classification task by processing point cloud data. In general, training a machine learning model using the determined point cloud augmentation policy enables the model to more effectively perform the perception task, i.e., by generating higher quality perception outputs. When deployed within an on-board system of a vehicle, the machine learning model can further enable the on-board system to generate better-informed planning decisions which in turn result in a safer journey, even when the vehicle is navigating through unconventional environments or inclement weathers such as rain or snow.

公开(公告)号：US20210192757A1

公开(公告)日：2021-06-24

申请号：US16726053

申请日：2019-12-23

Applicant: Waymo LLC

Inventor： Ruichi Yu ,  Sachithra Madhawa Hemachandra ,  Ian James Mahon ,  Congcong Li

IPC: G06T7/246 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for associating a new measurement of an object surrounding a vehicle with a maintained track. One of the methods includes receiving an object track for a particular object, receiving a new measurement characterizing a new object at a new time step, and determining whether the new object is the same as the particular object, comprising: generating a representation of the new object at the new and preceding time steps; generating a representation of the particular object at the new and preceding time steps; processing a first network input comprising the representations using a first neural network to generate an embedding of the first network input; and processing the embedding of the first network input using a second neural network to generate a predicted likelihood that the new object and the particular object are the same.

公开(公告)号：US20210150199A1

公开(公告)日：2021-05-20

申请号：US17099642

申请日：2020-11-16

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Jiyang Gao ,  Yukai Liu ,  Congcong Li ,  Zhishuai Zhang ,  Dragomir Anguelov

IPC: G06K9/00 ,  G06K9/32 ,  G06K9/62 ,  B60W30/095

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.

公开(公告)号：US12067471B2

公开(公告)日：2024-08-20

申请号：US17104921

申请日：2020-11-25

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Zijian Guo ,  Congcong Li ,  Xiaowei Li

IPC: G06F16/24 ,  B60W60/00 ,  G06F30/27 ,  G06N3/02 ,  G06N3/08 ,  G06V20/56 ,  G08G1/01

CPC classification number: G06N3/02 ,  B60W60/0027 ,  G06F30/27 ,  G06N3/08 ,  G06V20/56 ,  G08G1/0104 ,  B60W2554/4045 ,  B60W2554/4046

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and one or more embeddings of each sensor sample. Each sensor sample is generated from sensor data at multiple time steps and characterizes an environment at each of the multiple time steps. Each embedding corresponds to a respective portion of the sensor sample and has been generated by an embedding neural network. A query specifying a query portion of a query sensor sample is received. A query embedding corresponding to the query portion of the query sensor sample is generated through the embedding neural network. A plurality of relevant sensor samples that have embeddings that are closest to the query embedding are identified as characterizing similar scenarios to the query portion of the query sensor sample.

公开(公告)号：US11967103B2

公开(公告)日：2024-04-23

申请号：US17505900

申请日：2021-10-20

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06F18/214 ,  G06F18/25 ,  G06V20/58

CPC classification number: G06T7/73 ,  G06F18/214 ,  G06F18/251 ,  G06V20/58 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196 ,  G06T2207/30261

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US11861481B2

公开(公告)日：2024-01-02

申请号：US16726060

申请日：2019-12-23

Applicant: Waymo LLC

Inventor： Zijian Guo ,  Nichola Abdo ,  Junhua Mao ,  Congcong Li ,  Edward Stephen Walker, Jr.

IPC: G06F16/53 ,  G06N3/042 ,  G06F16/538 ,  G06F16/535 ,  G05D1/00 ,  G05D1/02 ,  G06N3/08 ,  G06N3/045

CPC classification number: G06N3/042 ,  G05D1/0088 ,  G05D1/0221 ,  G06F16/535 ,  G06F16/538 ,  G06N3/045 ,  G06N3/08 ,  G05D2201/0213

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and, for each sensor sample, an embedding of the sensor sample; receiving a request specifying a query sensor sample, wherein the query sensor sample characterizes a query environment region; and identifying, from the collection of sensor samples, a plurality of relevant sensor samples that characterize similar environment regions to the query environment region, comprising: processing the query sensor sample through the embedding neural network to generate a query embedding; and identifying, from sensor samples in a subset of the sensor samples in the collection, a plurality of sensor samples that have embeddings that are closest to the query embedding.

公开(公告)号：US11842282B2

公开(公告)日：2023-12-12

申请号：US17836287

申请日：2022-06-09

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Congcong Li ,  Yang Song

IPC: G06V20/56 ,  G06N3/084 ,  G05D1/02 ,  G06N3/08 ,  G06F18/20 ,  G06F18/25 ,  G06F18/21 ,  G06F18/214 ,  G06F18/2431 ,  G06N3/044 ,  G06V30/19 ,  G06V30/24 ,  G06V10/764 ,  G06V10/80 ,  G06V10/82 ,  G06V20/58

CPC classification number: G06N3/084 ,  G05D1/0221 ,  G05D1/0231 ,  G06F18/217 ,  G06F18/2148 ,  G06F18/2431 ,  G06F18/25 ,  G06F18/285 ,  G06N3/044 ,  G06N3/08 ,  G06V10/764 ,  G06V10/809 ,  G06V10/82 ,  G06V20/56 ,  G06V20/58 ,  G06V30/19173 ,  G06V30/2504

Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.

公开(公告)号：US11783568B2

公开(公告)日：2023-10-10

申请号：US17224763

申请日：2021-04-07

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Qian Yu ,  Congcong Li

IPC: G06N3/08 ,  G06V20/58 ,  G06V10/764 ,  G05D1/00 ,  G05D1/02 ,  G06F18/2413 ,  G06V10/82

CPC classification number: G06V10/764 ,  G05D1/0088 ,  G05D1/0221 ,  G05D1/0231 ,  G06F18/2413 ,  G06N3/08 ,  G06V10/82 ,  G06V20/58 ,  G05D2201/0213

Abstract: Some aspects of the subject matter disclosed herein include a system implemented on one or more data processing apparatuses. The system can include an interface configured to obtain, from one or more sensor subsystems, sensor data describing an environment of a vehicle, and to generate, using the sensor data, (i) one or more first neural network inputs representing sensor measurements for a particular object in the environment and (ii) a second neural network input representing sensor measurements for at least a portion of the environment that encompasses the particular object and additional portions of the environment that are not represented by the one or more first neural network inputs; and a convolutional neural network configured to process the second neural network input to generate an output, the output including a plurality of feature vectors that each correspond to a different one a plurality of regions of the environment.