公开(公告)号：US20240223919A1

公开(公告)日：2024-07-04

申请号：US18531154

申请日：2023-12-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Song Chen ,  Xinqiao Liu ,  Tsung-Hsun Tsai ,  Qing Chao ,  Michael Hall ,  David Maurice Moore ,  Sharvil Shailesh Talati

IPC: H04N25/616 ,  H04N25/131 ,  H04N25/53 ,  H04N25/67 ,  H04N25/771

CPC classification number: H04N25/616 ,  H04N25/131 ,  H04N25/53 ,  H04N25/67 ,  H04N25/771

Abstract: Systems and methods of the present disclosure include a digital image sensor pixel configured to perform digital correlated double-sampling (D-CDS) operations to compensate for fixed pattern noise (FPN) induced by comparator propagation delay variation across a digital pixel array. The pixel and D-CDS operations also support quantization of incident light using two pixel memory banks. The pixel includes data retention logic coupled to selectively lock and unlock the output of a comparator. The output of the comparator is combined with memory bank write enable signals to convert photodiode charge into digital pixel values stored in the two memory banks. The digital pixel values stored in the memory banks may be added or subtracted to generate an FPN compensated digital pixel value.

公开(公告)号：US11920916B1

公开(公告)日：2024-03-05

申请号：US17569480

申请日：2022-01-05

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael Hall ,  Xinqiao Liu

IPC: G01C3/08 ,  G01B11/22 ,  G01S7/4865 ,  H04N13/271

CPC classification number: G01B11/22 ,  G01S7/4865 ,  H04N13/271

Abstract: A depth camera assembly (DCA) includes a light generator emitting a beam of light into a local area and a detector. The detector captures light from the beam reflected by objects in the local area to a portion of an array of pixels that each include a single photon avalanche diode (SPAD). The location of the portion of the array is based in part on the angle of the beam emitted from the projector. The DCA identifies a set of pixels of the array corresponding to the portion and selectively retrieves current generated from the reflected light by the pixels in the portion of the array without retrieving current generated by pixels in other portions of the array.

公开(公告)号：US20230280468A1

公开(公告)日：2023-09-07

申请号：US17748777

申请日：2022-05-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Zhaoming Zhu ,  Michael Hall ,  Augusto Ronchini Ximenes ,  Shuochen Su ,  Charles Patrick Saunders ,  Gregory Cohoon

IPC: G01S17/894 ,  G02B27/00 ,  G01B11/22 ,  G01S7/4865 ,  G01S7/481 ,  G06T7/521 ,  G06F3/01

CPC classification number: G01S17/894 ,  G01B11/22 ,  G01S7/4815 ,  G01S7/4865 ,  G02B27/0093 ,  G06F3/013 ,  G06T7/521

Abstract: A projector for illuminating a target area is presented. The projector includes an array of emitters having a plurality of subarrays and an optical assembly. Each subarray includes one or more independently addressable channels emitting light in accordance with emission instructions. At least two of the subarrays are adjacent to each other and do not overlap. The optical assembly is configured to tile portions of the emitted light to form a light pattern for projection to a target area. The light pattern has a first plurality of sections and a second plurality of sections. Each section of the first plurality represents a first respective portion of the light pattern emitted from a corresponding subarray. Each section of the second plurality represents a second respective portion of the light pattern formed by tiling light emitted from two or more of the subarrays.

公开(公告)号：US11625845B2

公开(公告)日：2023-04-11

申请号：US17411405

申请日：2021-08-25

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Michael Hall

IPC: G06T7/521 ,  G06T19/00

Abstract: A depth measurement assembly (DMA) includes an illumination source that projects pulses of light (e.g., structured light) at a temporal pulsing frequency into a local area. The DMA includes a sensor that capture images of the pulses of light reflected from the local area and determines, using one or more of the captured images, one or more TOF phase shifts for the pulses of light. The DMA includes a controller coupled to the sensor and configured to determine a first set of estimated radial distances to an object in the local area based on the one or more TOF phase shifts. The controller determines a second estimated radial distance to the object based on an encoding of structured light and at least one of the captured images. The controller selects an estimated radial distance from the first set of radial distances.

公开(公告)号：US11587247B1

公开(公告)日：2023-02-21

申请号：US16838691

申请日：2020-04-02

Applicant: Meta Platforms Technologies, LLC

Inventor： Augusto Ronchini Ximenes ,  Michael Hall

IPC: G06T7/593 ,  H04N13/128 ,  G02B27/01

Abstract: A depth camera assembly (DCA) includes a direct time of flight system for determining depth information for a local area. The DCA includes an illumination source, a camera, and a controller. The illumination source projects light (e.g., pulse of light) into the local area. The camera detects reflections of the projected light from objects in the local area. The camera includes a detector where pixels are grouped into multiple macropixels that are coupled to an output bus. Specific macropixels from which information describing light detected by pixels in the specific macropixels is obtained. In come configurations, each macropixel includes a counter that is incremented based on detection of light by pixels in the macropixels. The counter may be used to select macropixels from which data is obtained.

公开(公告)号：US11924396B2

公开(公告)日：2024-03-05

申请号：US17579279

申请日：2022-01-19

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Michael Hall ,  Qing Chao ,  Byron Taylor ,  Xinqiao Liu

IPC: H04N13/271 ,  G01S17/89 ,  G02B26/08 ,  G02B27/01 ,  G02F1/1347 ,  G02F1/139 ,  G02F1/29 ,  G02F1/33 ,  G06F3/03 ,  G06T7/521 ,  H04N13/207 ,  H04N13/344 ,  H04N13/383 ,  G02B26/00

CPC classification number: H04N13/271 ,  G01S17/89 ,  G02B26/08 ,  G02B26/0808 ,  G02B27/0101 ,  G02B27/017 ,  G02F1/13471 ,  G02F1/1393 ,  G02F1/292 ,  G02F1/33 ,  G06F3/0325 ,  G06T7/521 ,  H04N13/207 ,  H04N13/344 ,  H04N13/383 ,  G02B26/004 ,  G02B2027/0138 ,  G02B2027/014 ,  G02F2201/305

Abstract: A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a transmitter, a receiver, and a controller. The transmitter illuminates a local area with outgoing light in accordance with emission instructions. The transmitter includes a fine steering element and a coarse steering element. The fine steering element deflects one or more optical beams at a first deflection angle to generate one or more first order deflected scanning beams. The coarse steering element deflects the one or more first order deflected scanning beams at a second deflection angle to generate the outgoing light projected into the local area. The receiver captures one or more images of the local area including portions of the outgoing light reflected from the local area. The controller determines depth information for one or more objects in the local area based in part on the captured one or more images.

公开(公告)号：US11922711B2

公开(公告)日：2024-03-05

申请号：US17234522

申请日：2021-04-19

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Michael Hall ,  Byron Taylor

IPC: G06V30/24 ,  G06F3/01 ,  G06F3/03 ,  G06F3/042 ,  G06F3/04883 ,  G06T7/73 ,  G06V20/20 ,  G06V40/10 ,  G06V40/16 ,  G06V40/18 ,  G06V40/20

CPC classification number: G06V30/2504 ,  G06T7/73 ,  G06V40/113 ,  G06V40/171 ,  G06V40/193 ,  G06F3/0425 ,  G06F3/04883

Abstract: Embodiments relate to tracking and determining a location of an object in an environment surrounding a user. A system includes one or more imaging devices and an object tracking unit. The system identifies an object in a search region, determines a tracking region that is smaller than the search region corresponding to the object, and scans the tracking region to determine a location associated with the object. The system may generate a ranking of objects, determine locations associated with the objects, and generate a model of the search region based on the locations associated with the objects.

公开(公告)号：US11774769B2

公开(公告)日：2023-10-03

申请号：US17173341

申请日：2021-02-11

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Michael Hall

IPC: G01B5/14 ,  G02B27/01 ,  G01B5/00 ,  G01B11/25 ,  G06T7/507 ,  H04N13/25 ,  G01B11/22 ,  G06T19/00

CPC classification number: G02B27/0176 ,  G01B5/0014 ,  G01B11/22 ,  G01B11/2513 ,  G06T7/507 ,  H04N13/25 ,  G02B2027/0138 ,  G06T19/006

Abstract: A depth measurement assembly (DMA) includes a pulsed illuminator assembly, a depth camera assembly, and a controller. The pulsed illuminator assembly has a structured light projector that projects pulses of structured light at a pulse rate into a local area. The depth camera assembly captures images data of an object in the local area illuminated with the pulses of structured light. An exposure interval of the depth camera assembly is pulsed and synchronized to the pulses projected by the pulsed illuminator assembly. The controller controls the pulsed illuminator assembly and the depth camera assembly so that they are synchronized. The controller also determine depth and/or tracking information of the object based on the captured image data. In some embodiments, the pulsed illuminator assembly have a plurality of structured light projectors that projects pulses of structured light at different times.

公开(公告)号：US20230260222A1

公开(公告)日：2023-08-17

申请号：US18087483

申请日：2022-12-22

Applicant: Meta Platforms Technologies, LLC

Inventor： Brevin Tilmon ,  Shuochen Su ,  Michael Hall

IPC: G06T19/00 ,  G06V10/74 ,  H04N13/128 ,  G01S17/89

CPC classification number: G06T19/006 ,  G06V10/761 ,  H04N13/128 ,  G01S17/89 ,  H04N2013/0081

Abstract: A binary depth mask model is trained for use in occlusion within a mixed reality application. The training leverages information about virtual object positions and depths that is inherently available to the system as part of rendering of virtual objects. Training is performed on a set of stereo images, a set of binary depth masks corresponding to the stereo images, and a depth value against which object depth is evaluated. Given this input, the training outputs the binary depth mask model, which when given a stereo image as input outputs a depth binary depth mask indicating which pixels of the stereo image are nearer, or father away, than the depth value. The depth mask model can be applied in real time to handle occlusion operations when compositing a given real-world stereo image with virtual objects.

公开(公告)号：US11716548B2

公开(公告)日：2023-08-01

申请号：US16298278

申请日：2019-03-11

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael Hall ,  Qing Chao

IPC: H04N25/53 ,  G01S17/89 ,  G02B27/01 ,  G01S7/4861 ,  H04N25/771 ,  H04N13/207

CPC classification number: H04N25/53 ,  G01S7/4861 ,  G01S17/89 ,  G02B27/017 ,  H04N25/771 ,  H04N13/207

Abstract: A depth camera assembly (DCA) determines distances between the DCA and objects in a local area within a field of view of the DCA. The DCA projects a series of sinusoidal patterns into the local area DCA and captures images of the sinusoidal patterns via a sensor. Each pixel of the augmented sensor includes a plurality of charge bins, and charge accumulated by a photodiode of a pixel during different time intervals (e.g., times when different sinusoidal patterns are emitted) is stored in a different charge storage bin. Charge may be retrieved from different charge storage bins to determine depth from the DCA.