Invention Grant
- Patent Title: Self-supervised training of a depth estimation model using depth hints
-
Application No.: US16864743Application Date: 2020-05-01
-
Publication No.: US11044462B2Publication Date: 2021-06-22
- Inventor: James Watson , Michael David Firman , Gabriel J. Brostow , Daniyar Turmukhambetov
- Applicant: Niantic, Inc.
- Applicant Address: US CA San Francisco
- Assignee: Niantic, Inc.
- Current Assignee: Niantic, Inc.
- Current Assignee Address: US CA San Francisco
- Agency: Fenwick & West LLP
- Main IPC: H04N13/268
- IPC: H04N13/268 ; G06T7/593 ; G06T7/80 ; G06T7/73 ; H04N13/00

Abstract:
A method for training a depth estimation model with depth hints is disclosed. For each image pair: for a first image, a depth prediction is determined by the depth estimation model and a depth hint is obtained; the second image is projected onto the first image once to generate a synthetic frame based on the depth prediction and again to generate a hinted synthetic frame based on the depth hint; a primary loss is calculated with the synthetic frame; a hinted loss is calculated with the hinted synthetic frame; and an overall loss is calculated for the image pair based on a per-pixel determination of whether the primary loss or the hinted loss is smaller, wherein if the hinted loss is smaller than the primary loss, then the overall loss includes the primary loss and a supervised depth loss between depth prediction and depth hint. The depth estimation model is trained by minimizing the overall losses for the image pairs.
Public/Granted literature
- US20200351489A1 SELF-SUPERVISED TRAINING OF A DEPTH ESTIMATION MODEL USING DEPTH HINTS Public/Granted day:2020-11-05
Information query