公开(公告)号：US11368662B2

公开(公告)日：2022-06-21

申请号：US17068243

申请日：2020-10-12

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Dan Lelescu ,  Jacques Duparre

IPC: H04N13/02 ,  H04N13/128 ,  H04N5/247 ,  G02B27/01 ,  H04N13/271 ,  G06T7/593 ,  H04N5/225 ,  H04N13/243 ,  G01C11/06 ,  H04N5/33 ,  H04N5/222 ,  H04N13/00

Abstract: Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.

公开(公告)号：US20210044790A1

公开(公告)日：2021-02-11

申请号：US17068243

申请日：2020-10-12

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Dan Lelescu ,  Jacques Duparre

IPC: H04N13/128 ,  H04N5/247 ,  G02B27/01 ,  H04N13/271 ,  G06T7/593 ,  H04N5/225 ,  H04N13/243 ,  G01C11/06

Abstract: Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.

公开(公告)号：US10909707B2

公开(公告)日：2021-02-02

申请号：US16537369

申请日：2019-08-09

Applicant: FotoNation Limited

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  H04N13/128 ,  H04N13/243 ,  G06T7/80 ,  G06T7/557 ,  G02B27/00 ,  G06T15/20 ,  H04N13/232 ,  H04N9/097 ,  H04N13/00

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US10818026B2

公开(公告)日：2020-10-27

申请号：US16685881

申请日：2019-11-15

Applicant: FotoNation Limited

Inventor： Ankit Jain ,  Priyam Chatterjee ,  Kartik Venkataraman

IPC: G06T7/593 ,  G06T7/44 ,  G06T7/13 ,  G06T7/11 ,  G06T7/194 ,  G06T5/30 ,  G06T7/136 ,  G06T5/50

Abstract: Systems and methods for hybrid depth regularization in accordance with various embodiments of the invention are disclosed. In one embodiment of the invention, a depth sensing system comprises a plurality of cameras; a processor; and a memory containing an image processing application. The image processing application may direct the processor to obtain image data for a plurality of images from multiple viewpoints, the image data comprising a reference image and at least one alternate view image; generate a raw depth map using a first depth estimation process, and a confidence map; and generate a regularized depth map. The regularized depth map may be generated by computing a secondary depth map using a second different depth estimation process; and computing a composite depth map by selecting depth estimates from the raw depth map and the secondary depth map based on the confidence map.

公开(公告)号：US20190356863A1

公开(公告)日：2019-11-21

申请号：US16529522

申请日：2019-08-01

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis ,  Jacques Duparre ,  Shane Ching-Feng Hu

IPC: H04N5/247 ,  H04N13/239 ,  H04N5/232 ,  H04N5/225 ,  G06T7/557 ,  G02B13/00 ,  G02B3/00 ,  H04N5/365 ,  G02B5/20 ,  G06T11/60 ,  H04N5/265 ,  H04N5/262 ,  H04N9/73 ,  H04N9/09 ,  H04N13/128 ,  G06T7/50 ,  G06T19/20 ,  H04N9/097 ,  H04N9/04 ,  H04N5/357 ,  H04N5/355 ,  H04N5/349 ,  H04N5/341 ,  H04N5/33

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

公开(公告)号：US10455218B2

公开(公告)日：2019-10-22

申请号：US15791242

申请日：2017-10-23

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Paul Gallagher ,  Ankit Jain ,  Semyon Nisenzon

IPC: H04N13/243 ,  G01P3/38 ,  G06T7/593 ,  G06T7/285 ,  G06T7/557 ,  G06T7/55 ,  G06T7/579

Abstract: Systems and methods for stereo imaging with camera arrays in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating depth information for an object using two or more array cameras that each include a plurality of imagers includes obtaining a first set of image data captured from a first set of viewpoints, identifying an object in the first set of image data, determining a first depth measurement, determining whether the first depth measurement is above a threshold, and when the depth is above the threshold: obtaining a second set of image data of the same scene from a second set of viewpoints located known distances from one viewpoint in the first set of viewpoints, identifying the object in the second set of image data, and determining a second depth measurement using the first set of image data and the second set of image data.

公开(公告)号：US10375319B2

公开(公告)日：2019-08-06

申请号：US16199566

申请日：2018-11-26

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Amandeep S. Jabbi ,  Robert H. Mullis ,  Jacques Duparre ,  Shane Ching-Feng Hu

IPC: H04N5/365 ,  H04N5/225 ,  H04N5/247 ,  G02B13/00 ,  G02B5/20 ,  H04N13/128 ,  H04N5/232 ,  H04N5/33 ,  H04N5/341 ,  H04N5/349 ,  H04N5/355 ,  H04N5/357 ,  H04N9/04 ,  H04N9/097 ,  G06T19/20 ,  H04N9/09 ,  H04N9/73 ,  H04N5/262 ,  H04N5/265 ,  G06T11/60 ,  G02B3/00 ,  G06T7/557 ,  H04N13/239 ,  G06T7/50 ,  H04N13/257 ,  H04N13/00

Abstract: Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. An imaging device in accordance with one embodiment of the invention includes at least one imager array, and each imager in the array comprises a plurality of light sensing elements and a lens stack including at least one lens surface, where the lens stack is configured to form an image on the light sensing elements, control circuitry configured to capture images formed on the light sensing elements of each of the imagers, and a super-resolution processing module configured to generate at least one higher resolution super-resolved image using a plurality of the captured images.

公开(公告)号：US10334241B2

公开(公告)日：2019-06-25

申请号：US15797126

申请日：2017-10-30

Applicant: FotoNation Limited

Inventor： Jacques Duparre ,  Andrew Kenneth John McMahon ,  Dan Lelescu ,  Kartik Venkataraman ,  Gabriel Molina

IPC: H04N17/00 ,  H04N5/225 ,  H04N5/367 ,  H04N5/217

Abstract: Systems and methods for detecting defective camera arrays, optic arrays and/or sensors are described. One embodiment includes capturing image data using a camera array; dividing the captured images into a plurality of corresponding image regions; identifying the presence of localized defects in any of the cameras by evaluating the image regions in the captured images; and detecting a defective camera array using the image processing system when the number of localized defects in a specific set of image regions exceeds a predetermined threshold, where the specific set of image regions is formed by: a common corresponding image region from at least a subset of the captured images; and any additional image region in a given image that contains at least one pixel located within a predetermined maximum parallax shift distance along an epipolar line from a pixel within said common corresponding image region within the given image.

公开(公告)号：US20190098209A1

公开(公告)日：2019-03-28

申请号：US16184840

申请日：2018-11-08

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Jacques Duparré

IPC: H04N5/232 ,  H04N13/243 ,  G06T7/557 ,  H04N5/247 ,  H04N5/225 ,  G06T3/40 ,  H04N5/369 ,  H04N17/00 ,  H04N5/265 ,  G02B3/00 ,  G02B13/02 ,  H04N13/232 ,  H04N13/00 ,  H04N13/257

CPC classification number: H04N5/23232 ,  G02B3/0056 ,  G02B3/0062 ,  G02B13/02 ,  G06T3/4053 ,  G06T7/557 ,  G06T2207/10052 ,  H04N5/2258 ,  H04N5/23238 ,  H04N5/23296 ,  H04N5/247 ,  H04N5/265 ,  H04N5/3696 ,  H04N13/232 ,  H04N13/243 ,  H04N13/257 ,  H04N17/002 ,  H04N2013/0081

Abstract: Systems and methods for implementing array camera configurations that include a plurality of constituent array cameras, where each constituent array camera provides a distinct field of view and/or a distinct viewing direction, are described. In several embodiments, image data captured by the constituent array cameras is used to synthesize multiple images that are subsequently blended. In a number of embodiments, the blended images include a foveated region. In certain embodiments, the blended images possess a wider field of view than the fields of view of the multiple images.

公开(公告)号：US10235590B2

公开(公告)日：2019-03-19

申请号：US15865208

申请日：2018-01-08

Applicant: FotoNation Limited

Inventor： Kartik Venkataraman ,  Semyon Nisenzon ,  Dan Lelescu

IPC: G06K9/36 ,  H04N13/128 ,  H04N13/161 ,  H04N13/243 ,  H04N13/271 ,  G06T9/00 ,  G06T9/20 ,  H04N19/597 ,  H04N19/625 ,  H04N19/136 ,  G06T3/40 ,  H04N19/85 ,  G06T15/08 ,  G06T7/50 ,  G06T7/593 ,  H04N13/00

Abstract: Systems and methods for storing images synthesized from light field image data and metadata describing the images in electronic files in accordance with embodiments of the invention are disclosed. One embodiment includes a processor and memory containing an encoding application and light field image data, where the light field image data comprises a plurality of low resolution images of a scene captured from different viewpoints. In addition, the encoding application configures the processor to synthesize a higher resolution image of the scene from a reference viewpoint using the low resolution images, where synthesizing the higher resolution image involves creating a depth map that specifies depths from the reference viewpoint for pixels in the higher resolution image; encode the higher resolution image; and create a light field image file including the encoded image, the low resolution images, and metadata including the depth map.