公开(公告)号：US20220254105A1

公开(公告)日：2022-08-11

申请号：US17652078

申请日：2022-02-22

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman

IPC: G06T17/20 ,  G06T7/149 ,  G06V40/16 ,  G06T7/593

Abstract: In an embodiment, a 3D facial modeling system includes a plurality of cameras configured to capture images from different viewpoints, a processor, and a memory containing a 3D facial modeling application and parameters defining a face detector, wherein the 3D facial modeling application directs the processor to obtain a plurality of images of a face captured from different viewpoints using the plurality of cameras, locate a face within each of the plurality of images using the face detector, wherein the face detector labels key feature points on the located face within each of the plurality of images, determine disparity between corresponding key feature points of located faces within the plurality of images, and generate a 3D model of the face using the depth of the key feature points.

公开(公告)号：US11412158B2

公开(公告)日：2022-08-09

申请号：US16907016

申请日：2020-06-19

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis ,  Jacques Duparre ,  Shane Ching-Feng Hu

IPC: H04N5/262 ,  H04N5/225 ,  H04N5/247 ,  G02B13/00 ,  G02B5/20 ,  H04N5/365 ,  H04N13/128 ,  H04N5/232 ,  H04N5/33 ,  H04N5/341 ,  H04N5/349 ,  H04N5/355 ,  H04N5/357 ,  G06T7/557 ,  H04N13/239 ,  G06T7/50 ,  H04N9/097 ,  G06T19/20 ,  H04N9/09 ,  H04N9/73 ,  H04N5/265 ,  G06T11/60 ,  G02B3/00 ,  H04N13/257 ,  H04N9/04 ,  H04N13/00

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

公开(公告)号：US20210358154A1

公开(公告)日：2021-11-18

申请号：US17331428

申请日：2021-05-26

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman

IPC: G06T7/55 ,  G06T7/70 ,  G06N3/04

Abstract: Systems and methods for depth estimation in accordance with embodiments of the invention are illustrated. One embodiment includes a method for estimating depth from images. The method includes steps for receiving a plurality of source images captured from a plurality of different viewpoints using a processing system configured by an image processing application, generating a target image from a target viewpoint that is different to the viewpoints of the plurality of source images based upon a set of generative model parameters using the processing system configured by the image processing application, and identifying depth information of at least one output image based on the predicted target image using the processing system configured by the image processing application.

公开(公告)号：US20210133927A1

公开(公告)日：2021-05-06

申请号：US17098432

申请日：2020-11-15

Applicant: FotoNation Limited

Inventor： Dan Lelescu ,  Gabriel Molina ,  Kartik Venkataraman

IPC: G06T3/40 ,  H04N13/106 ,  H04N13/111 ,  H04N13/128 ,  H04N13/139 ,  G06T5/00 ,  H04N5/232 ,  G06T5/50

Abstract: Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.

公开(公告)号：US10674138B2

公开(公告)日：2020-06-02

申请号：US16179796

申请日：2018-11-02

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Paul Gallagher ,  Ankit K. Jain ,  Semyon Nisenzon ,  Dan Lelescu ,  Florian Ciurea ,  Gabriel Molina

IPC: H04N13/271 ,  G01P3/38 ,  H04N13/243 ,  H04N13/232

Abstract: Systems with an array camera augmented with a conventional camera in accordance with embodiments of the invention are disclosed. In some embodiments, the array camera is used to capture a first set of image data of a scene and a conventional camera is used to capture a second set of image data for the scene. An object of interest is identified in the first set of image data. A first depth measurement for the object of interest is determined and compared to a predetermined threshold. If the first depth measurement is above the threshold, a second set of image data captured using the conventional camera is obtained. The object of interest is identified in the second set of image data and a second depth measurement for the object of interest is determined using at least a portion of the first set of image data and at least a portion of the second set of image data.

公开(公告)号：US10540806B2

公开(公告)日：2020-01-21

申请号：US15898666

申请日：2018-02-19

Applicant: FotoNation Limited

Inventor： Samuel Yang ,  Manohar Srikanth ,  Dan Lelescu ,  Kartik Venkataraman

IPC: G06T15/20 ,  G06K9/00 ,  G06T5/00 ,  G06T5/50

Abstract: Systems and methods for automatically correcting apparent distortions in close range photographs that are captured using an imaging system capable of capturing images and depth maps are disclosed. In many embodiments, faces are automatically detected and segmented from images using a depth-assisted alpha matting. The detected faces can then be re-rendered from a more distant viewpoint and composited with the background to create a new image in which apparent perspective distortion is reduced.

公开(公告)号：US10380752B2

公开(公告)日：2019-08-13

申请号：US15858974

申请日：2017-12-29

Applicant: FotoNation Limited

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/80 ,  G02B27/00 ,  G06T15/20 ,  G06T7/557 ,  G06T7/593 ,  H04N13/00 ,  H04N9/097 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US20190109998A1

公开(公告)日：2019-04-11

申请号：US16199566

申请日：2018-11-26

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis ,  Jacques Duparre ,  Shane Ching-Feng Hu

IPC: H04N5/247 ,  H04N13/128 ,  H04N5/365 ,  G02B5/20 ,  G06T7/50 ,  H04N13/239 ,  G06T7/557 ,  G02B13/00 ,  H04N5/225 ,  H04N5/232 ,  H04N5/33 ,  H04N5/341 ,  H04N5/349 ,  H04N5/355 ,  H04N5/357 ,  H04N9/04 ,  H04N9/097 ,  G06T19/20 ,  H04N9/09 ,  H04N9/73 ,  H04N5/262 ,  H04N5/265 ,  G06T11/60 ,  G02B3/00 ,  H04N13/00 ,  H04N13/257

CPC classification number: H04N5/247 ,  G02B3/0056 ,  G02B3/0062 ,  G02B5/20 ,  G02B5/201 ,  G02B13/0015 ,  G06T7/50 ,  G06T7/557 ,  G06T11/60 ,  G06T19/20 ,  G06T2200/04 ,  G06T2207/10028 ,  G06T2207/10052 ,  G06T2207/20221 ,  H04N5/2253 ,  H04N5/2254 ,  H04N5/2258 ,  H04N5/23232 ,  H04N5/23238 ,  H04N5/23296 ,  H04N5/262 ,  H04N5/265 ,  H04N5/332 ,  H04N5/3415 ,  H04N5/349 ,  H04N5/35545 ,  H04N5/357 ,  H04N5/365 ,  H04N9/045 ,  H04N9/09 ,  H04N9/097 ,  H04N9/735 ,  H04N13/128 ,  H04N13/239 ,  H04N13/257 ,  H04N2013/0081

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

公开(公告)号：US20190089947A1

公开(公告)日：2019-03-21

申请号：US16179796

申请日：2018-11-02

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Paul Gallagher ,  Ankit K. Jain ,  Semyon Nisenzon ,  Dan Lelescu ,  Florian Ciurea ,  Gabriel Molina

IPC: H04N13/271 ,  G01P3/38 ,  H04N13/243 ,  H04N13/232

CPC classification number: H04N13/271 ,  G01P3/38 ,  H04N13/232 ,  H04N13/243

Abstract: Systems with an array camera augmented with a conventional camera in accordance with embodiments of the invention are disclosed. In some embodiments, the array camera is used to capture a first set of image data of a scene and a conventional camera is used to capture a second set of image data for the scene. An object of interest is identified in the first set of image data. A first depth measurement for the object of interest is determined and compared to a predetermined threshold. If the first depth measurement is above the threshold, a second set of image data captured using the conventional camera is obtained. The object of interest is identified in the second set of image data and a second depth measurement for the object of interest is determined using at least a portion of the first set of image data and at least a portion of the second set of image data.

公开(公告)号：US20230007223A1

公开(公告)日：2023-01-05

申请号：US17807508

申请日：2022-06-17

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Dan Lelescu ,  Jacques Duparre

IPC: H04N13/128 ,  H04N5/247 ,  G02B27/01 ,  H04N13/271 ,  G06T7/593 ,  H04N5/225 ,  H04N13/243 ,  G01C11/06

Abstract: Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.