公开(公告)号：US11509803B1

公开(公告)日：2022-11-22

申请号：US17077826

申请日：2020-10-22

Applicant: Meta Platforms Technologies, LLC

Inventor： Fengqiang Li ,  Michael Hall

IPC: H04N5/225 ,  H04N5/341 ,  H04N5/232 ,  H04N5/222 ,  G02B27/01

Abstract: A camera assembly for determining depth information for a local area includes a light source assembly, a camera assembly, and a controller. The light source assembly projects pulses of light into the local area. The camera assembly images a portion of the local area illuminated with the pulses. The camera assembly includes augmented pixels, each augmented pixel having a plurality of gates and at least some of the gates have a respective local storage location. An exposure interval of each augmented pixel is divided into intervals associated with the gates, and each local storage location stores image data during a respective interval. The controller reads out, after the exposure interval of each augmented pixel, the image data stored in the respective local storage locations of each augmented pixel to generate image data frames. The controller determines depth information for the local area based in part on the image data frames.

公开(公告)号：US11480684B2

公开(公告)日：2022-10-25

申请号：US17138537

申请日：2020-12-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Augusto Ronchini Ximenes ,  Michael Hall ,  Zhaoming Zhu ,  Shuochen Su

IPC: G01S17/894 ,  G02B27/01 ,  G01S17/86 ,  G01S7/4863

Abstract: A direct time of flight depth system includes an addressable illumination source, an active depth sensor, and a controller. The addressable illumination source includes an array of emitters and an optical assembly that is used to generate an array of dots emitted into a local area. Each emitter is independently addressable, allowing selective illumination of different portions of the local area. The addressable illumination source is aligned with the active depth sensor so each dot maps to a corresponding macropixel (e.g., 4×4 array of SPADs) on the active depth sensor. Data from the active depth sensor is readout and used to determine depth information.

公开(公告)号：US11601607B2

公开(公告)日：2023-03-07

申请号：US17069709

申请日：2020-10-13

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Rajesh Lachhmandas Chhabria ,  Shuochen Su ,  Monica Stewart ,  David Vakrat ,  Shengqiong Xie ,  Michael Hall

IPC: H04N5/33 ,  G02B27/00 ,  G02B27/01

Abstract: An electronic device includes one or more processors and memory storing instructions for execution by the one or more processors. The stored instructions include instructions for: receiving infrared image information for a three-dimensional area; receiving non-infrared image information for the same three-dimensional area; performing nonlinear intensity adjustment for the received infrared image information; performing nonlinear intensity adjustment for the received non-infrared image information; blending the intensity-adjusted infrared image information and the intensity-adjusted non-infrared image information to obtain a merged image information; and providing the merged image information for determining a depth map. Also disclosed are a corresponding method performed by the electronic device and a computer readable storage medium storing instructions for execution by one or more processors of an electronic device.

公开(公告)号：US11417005B1

公开(公告)日：2022-08-16

申请号：US17203049

申请日：2021-03-16

Applicant: Meta Platforms Technologies, LLC

Inventor： Qing Chao ,  Michael Hall

IPC: G06T7/00 ,  G06F1/00 ,  G06T7/521 ,  G02B26/08 ,  G02B5/18 ,  H04N5/225 ,  G06F1/16 ,  G02B26/10 ,  G02B5/30

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a polarized light generator, an imaging device, and a controller. The polarized light generator modulates one or more optical beams emitted from an illumination source to generate modulated light, and projects the modulated light into the local area as polarized light having a first polarization. The imaging device receives light from the local area, the received light including ambient light and a portion of the polarized light reflected from the local area. The imaging device reduces an intensity of the received light having polarization different from a second polarization to generate filtered light substantially composed of light of the second polarization, and detects the portion of the polarized light having the second polarization using the filtered light. The controller determines depth information for the local area based on the detected portion of the polarized light.

公开(公告)号：US12026906B1

公开(公告)日：2024-07-02

申请号：US18096219

申请日：2023-01-12

Applicant: Meta Platforms Technologies, LLC

Inventor： Augusto Ronchini Ximenes ,  Michael Hall

IPC: G06T7/593 ,  G02B27/01 ,  H04N13/128

CPC classification number: G06T7/593 ,  G02B27/0172 ,  H04N13/128 ,  G02B2027/0178 ,  G06T2207/10028

Abstract: Each pixel of a plurality of pixels captures a portion of outgoing light illuminating a local area reflected from the one or more objects in the local area. A flag is generated via a flag determination logic circuit coupled to each pixel of the plurality of pixels. The flag indicates whether the reflected light was captured within a threshold amount of time from a current time. A time to digital converter is enabled via a flag determination logic circuit. The time to digital converter is associated with each pixel of the plurality of pixels to generate a digital representation of time when each pixel of the plurality of pixels captured the reflected light. Depth information is determined for the one or more objects in the local area based in part on the generated flag and the digital representation of time.

公开(公告)号：US20230328401A1

公开(公告)日：2023-10-12

申请号：US18208143

申请日：2023-06-09

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael Hall ,  Qing Chao

IPC: H04N25/53 ,  G01S17/89 ,  G02B27/01 ,  G01S7/4861 ,  H04N25/771

CPC classification number: H04N25/53 ,  G01S17/89 ,  G02B27/017 ,  G01S7/4861 ,  H04N25/771 ,  H04N13/207

Abstract: A depth camera assembly (DCA) determines distances between the DCA and objects in a local area within a field of view of the DCA. The DCA projects a series of sinusoidal patterns into the local area DCA and captures images of the sinusoidal patterns via a sensor. Each pixel of the augmented sensor includes a plurality of charge bins, and charge accumulated by a photodiode of a pixel during different time intervals (e.g., times when different sinusoidal patterns are emitted) is stored in a different charge storage bin. Charge may be retrieved from different charge storage bins to determine depth from the DCA.

公开(公告)号：US11740075B2

公开(公告)日：2023-08-29

申请号：US17504004

申请日：2021-10-18

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Michael Hall ,  Xinqiao Liu ,  Zhaoming Zhu ,  Rajesh Lachhmandas Chhabria ,  Huixuan Tang ,  Shuochen Su ,  Zihe Gao

IPC: G06T7/521 ,  G01B11/22 ,  G01B11/25 ,  G06T7/529 ,  G02B27/01 ,  G06T7/11 ,  G06T7/174 ,  G06T7/55 ,  G06T7/00 ,  G06T7/90 ,  H04N13/106 ,  H04N13/204 ,  G06T7/593 ,  H04N13/128 ,  H04N13/271 ,  H04N13/239 ,  H04N23/56 ,  G06V10/22 ,  G06V10/145 ,  H04N13/00

CPC classification number: G01B11/2513 ,  G01B11/22 ,  G02B27/0172 ,  G06T7/11 ,  G06T7/174 ,  G06T7/521 ,  G06T7/529 ,  G06T7/55 ,  G06T7/593 ,  G06T7/90 ,  G06T7/97 ,  G06V10/145 ,  G06V10/22 ,  H04N13/106 ,  H04N13/128 ,  H04N13/204 ,  H04N13/239 ,  H04N13/271 ,  H04N23/56 ,  G02B2027/014 ,  G02B2027/0138 ,  G02B2027/0178 ,  G06T2207/10028 ,  H04N2013/0081

Abstract: A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.

公开(公告)号：US11637978B1

公开(公告)日：2023-04-25

申请号：US17551162

申请日：2021-12-14

Applicant: Meta Platforms Technologies, LLC

Inventor： Charles Patrick Saunders ,  Michael Hall ,  Augusto Ronchini Ximenes

IPC: H04N5/369 ,  H04N5/345 ,  H04N5/225

Abstract: A depth camera assembly (DCA) includes a direct time of flight system for determining depth information for a local area. The DCA includes an illumination source, a camera, and a controller. In some embodiments, the controller uses previous image frames to determine confidence measurements, and selectively adjusts a number of pulses from the illuminator in a subsequent frame based on the determined confidence values. In some embodiments, the sensor uses autonomous gating, and the depth system includes a depth recovery pipeline which provide depth map estimates from the autonomous gated measurements.

公开(公告)号：US11567318B1

公开(公告)日：2023-01-31

申请号：US16135452

申请日：2018-09-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael Hall ,  Xinqiao Liu

IPC: G02B27/01 ,  G06F3/01 ,  G01B11/22 ,  G06T19/00 ,  G06V40/18

Abstract: An eye monitoring system is included in a headset of a virtual reality system or of an augmented reality system. The eye monitoring system determines distances between the eye monitoring system and portions of a user's eye enclosed by the headset. The eye monitoring system projects a temporally periodic pattern of light onto the user's eye via a sensor. The eye monitoring system determines a distance between the eye monitoring system and locations of the user's eye based on a phase shift of the periodic pattern of light captured by each pixel of the sensor. From the determined distances, the eye monitoring system determines features of the user's eye.

公开(公告)号：US12146965B1

公开(公告)日：2024-11-19

申请号：US18100912

申请日：2023-01-24

Applicant: Meta Platforms Technologies, LLC

Inventor： Augusto Ronchini Ximenes ,  Michael Hall

IPC: G01S17/00 ,  G01S7/48 ,  G01S7/483 ,  G01S7/4863 ,  G01S17/08 ,  G01S17/894 ,  G02B27/01 ,  G06F3/01 ,  G06T7/50 ,  G06T19/20 ,  H04N13/254 ,  G01S7/4865 ,  G01S17/88 ,  G04F10/00 ,  H04N13/00 ,  H04S7/00

Abstract: A depth camera assembly (DCA) includes a direct time of flight system for determining depth information for a local area. The DCA includes an illumination source, a camera, and a controller. The illumination source projects light (e.g., pulse of light) into the local area. The camera detects reflections of the projected light from objects in the local area. Using an internal gating selection procedure, the controller selects a gate window that is likely to be associated with reflection of a pulse of light from an object. The selected gate may be used for depth determination. The internal gating selection procedures may be achieved through external target location and selection or through internal self-selection.