公开(公告)号：US09344638B2

公开(公告)日：2016-05-17

申请号：US14292792

申请日：2014-05-30

Applicant: Apple Inc.

Inventor： Xuemei Zhang ,  Marius Tico ,  Yingjun Bai ,  Farhan A. Baqai ,  Fabio Riccardi ,  Juanqing Chen ,  Paul M. Hubel ,  George E. Williams ,  Todd S. Sachs ,  Yongfang Liang

IPC: H04N5/225 ,  H04N5/235 ,  G03B7/00 ,  H04N3/14 ,  H04N5/335 ,  H04N5/228 ,  H04N5/91 ,  G06K9/46 ,  G06T5/00 ,  H04N5/265 ,  H04N5/232 ,  G06T5/40

CPC classification number: H04N5/2355 ,  G06K9/4642 ,  G06K9/4661 ,  G06T5/001 ,  G06T5/009 ,  G06T5/40 ,  G06T2207/10144 ,  G06T2207/20016 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/23245 ,  H04N5/265 ,  H04N5/91

Abstract: Systems, methods, and computer readable media to capture and process high dynamic range (HDR) images when appropriate for a scene are disclosed. When appropriate, multiple images at a single—slightly underexposed—exposure value are captured (making a constant bracket HDR capture sequence) and local tone mapping (LTM) applied to each image. Local tone map and histogram information can be used to generate a noise-amplification mask which can be used during fusion operations. Images obtained and fused in the disclosed manner provide high dynamic range with improved noise and de-ghosting characteristics.

Abstract translation: 公开了适用于场景时捕获和处理高动态范围（HDR）图像的系统，方法和计算机可读介质。 在适当的情况下，捕获单个稍微曝光不足的曝光值的多个图像（使恒定的支架HDR捕获序列）和本地色调映射（LTM）应用于每个图像。 本地色调图和直方图信息可用于生成可在融合操作期间使用的噪声放大掩模。 以所公开的方式获得和融合的图像提供高动态范围，具有改进的噪声和去重影特性。

公开(公告)号：US20150070537A1

公开(公告)日：2015-03-12

申请号：US14269941

申请日：2014-05-05

Applicant: Apple Inc.

Inventor： Yingjun Bai ,  Michael Chin ,  Touraj Tajbakhsh

IPC: H04N9/73 ,  G06T7/40

CPC classification number: H04N9/735 ,  H04N1/60 ,  H04N1/6086 ,  H04N5/2353 ,  H04N5/3572

Abstract: This disclosure pertains to systems, methods, and computer readable media for performing lens shading correction (LSC) operations that modulate gains based on scene lux level and lens focus distance. These gains compensate for both color lens shading (i.e., the deviation between R, G, and B channels) and vignetting (i.e., the drop off in pixel intensity around the edges of an image). As scene illuminance increases, the sensor captures more signal from the actual scene, and the lens shading effects begin to appear. To deal with the situation, the lens shading gains are configured to adaptively ‘scale down’ when scene lux approaches zero and ‘scale up’ when scene lux changes from near zero to become larger. The lens shading gain may also be modulated based on the focus distance. For optical systems without zoom, the inventors have discovered that the amount of lens shading fall off changes as focus distance changes.

Abstract translation: 本公开涉及用于执行基于场景勒克斯水平和透镜焦距的调制增益的透镜阴影校正（LSC）操作的系统，方法和计算机可读介质。 这些增益补偿了彩色透镜阴影（即，R，G和B通道之间的偏差）和渐晕（即，围绕图像边缘的像素强度的下降）。 随着场景照度的增加，传感器捕获来自实际场景的更多信号，并开始出现镜头遮光效果。 为了处理这种情况，当场景勒克斯接近零时，镜头遮蔽增益被配置为自适应地“缩小”，当场景勒克斯从近零变化到更大时，“镜像”增大。 透镜遮光增益也可以基于焦距进行调制。 对于没有变焦的光学系统，本发明人已经发现，随着焦距的变化，透镜遮蔽量的下降变化。

公开(公告)号：US20220237887A1

公开(公告)日：2022-07-28

申请号：US17659377

申请日：2022-04-15

Applicant: Apple Inc.

Inventor： Vignesh Jagadeesh ,  Yingjun Bai ,  Guillaume Tartavel ,  Gregory Guyomarc'h

IPC: G06V10/46 ,  G06K9/62 ,  G06V10/22

Abstract: Various methods for utilizing a saliency heatmaps are described. The methods include obtaining image data corresponding to an image of a scene, obtaining a saliency heatmap for the image of the scene based on a saliency network, wherein the saliency heatmap indicates a likelihood of saliency for a corresponding portion of the scene, and manipulating the image data based on the saliency heatmap. In embodiments, the saliency heatmap may be produced using a trained machine learning model. The saliency heatmap may be used for various image processing tasks, such as determining which portion(s) of a scene to base an image capture device's autofocus, auto exposure, and/or white balance operations upon. According to some embodiments, one or more bounding boxes may be generated based on the saliency heatmap, e.g., using an optimization operation, which bounding box(es) may be used to assist or enhance the performance of various image processing tasks.

公开(公告)号：US11195247B1

公开(公告)日：2021-12-07

申请号：US16777074

申请日：2020-01-30

Applicant: Apple Inc.

Inventor： Henryk K. Blasinski ,  Xuemei Zhang ,  Yingjun Bai

IPC: G06T7/00 ,  G06T1/00 ,  G06T5/50 ,  G06T5/00 ,  G06T5/40

Abstract: An electronic image capture device captures a first image of a scene at a first time. A first local tone mapping operator for a first portion of the first image is determined. The electronic image capture device further captures a second image of the scene at a second time. A motion of the electronic device between the first time and the second time is determined. A second local tone mapping operator for a second portion of the second image is determined. The second portion is determined to correspond to the first portion based, at least in part, on the determined motion of the electronic device. The second local tone mapping operator is determined based, at least in part, on the first local tone mapping operator. At least the second local tone mapping operator is applied to the second portion of the second image.

公开(公告)号：US10205967B2

公开(公告)日：2019-02-12

申请号：US15070620

申请日：2016-03-15

Applicant: Apple Inc.

Inventor： Hao Pan ,  Yingjun Bai ,  Xuemei Zhang ,  Haitao Guo ,  Munehiro Nakazato ,  Xiaohua Yang

IPC: H04N19/86 ,  H04N19/12 ,  H04N19/156 ,  H04N19/186 ,  H04N19/44

Abstract: This disclosure relates to a wide gamut encoder capable of receiving a wide gamut color image in accordance with a wide gamut standard. The encoder can encode one or more wide gamut color image pixel values into portions of narrow gamut encoding elements for transmission to a video encoder. The encoder can implement an advanced extended YCC format that is backward compatible with a P3 color gamut.

公开(公告)号：US20180070068A1

公开(公告)日：2018-03-08

申请号：US15697228

申请日：2017-09-06

Applicant: Apple Inc.

Inventor： Yonghui Zhao ,  Xuemei Zhang ,  Yingjun Bai ,  Gregory Guyomarc'h ,  Kosta Dabov

IPC: H04N9/73 ,  H04N5/232 ,  H04N5/235 ,  H04N5/357 ,  H04N5/367 ,  H04N9/68

CPC classification number: H04N9/735 ,  H04N5/23212 ,  H04N5/2357 ,  H04N5/3572 ,  H04N5/367 ,  H04N9/045 ,  H04N9/68

Abstract: Techniques for auto white balancing of captured images based on detection of flicker in ambient light is described. When flicker is detected in ambient light during an image capture event, and the flicker is unchanging during the image capture event, a white point of image data may be estimated according to a first technique. When flicker is detected in ambient light during an image capture event, and the flicker is changing during the image capture event, a white point of image data may be estimated according to a second technique. When flicker is not detected, a white point of image data may be estimated according to a third technique. Image data may be color corrected based on the estimated white point.

公开(公告)号：US20170289512A1

公开(公告)日：2017-10-05

申请号：US15626629

申请日：2017-06-19

Applicant: Apple Inc.

Inventor： Yingjun Bai ,  Michael Chin ,  Touraj Tajbakhsh

IPC: H04N9/73 ,  H04N1/60 ,  H04N5/357 ,  H04N5/235

CPC classification number: H04N9/735 ,  H04N1/60 ,  H04N1/6086 ,  H04N5/2353 ,  H04N5/3572

Abstract: This disclosure pertains to systems, methods, and computer readable media for performing lens shading correction (LSC) operations that modulate gains based on scene lux level and lens focus distance. These gains compensate for both color lens shading (i.e., the deviation between R, G, and B channels) and vignetting (i.e., the drop off in pixel intensity around the edges of an image). As scene illuminance increases, the sensor captures more signal from the actual scene, and the lens shading effects begin to appear. To deal with the situation, the lens shading gains are configured to adaptively ‘scale down’ when scene lux approaches zero and ‘scale up’ when scene lux changes from near zero to become larger. The lens shading gain may also be modulated based on the focus distance. For optical systems without zoom, the inventors have discovered that the amount of lens shading fall off changes as focus distance changes.

公开(公告)号：US09699428B2

公开(公告)日：2017-07-04

申请号：US14269941

申请日：2014-05-05

Applicant: Apple Inc.

Inventor： Yingjun Bai ,  Michael Chin ,  Touraj Tajbakhsh

IPC: H04N5/208 ,  H04N9/73 ,  H04N1/60 ,  H04N5/235 ,  H04N5/357 ,  H04N9/64

CPC classification number: H04N9/735 ,  H04N1/60 ,  H04N1/6086 ,  H04N5/2353 ,  H04N5/3572

Abstract: This disclosure pertains to systems, methods, and computer readable media for performing lens shading correction (LSC) operations that modulate gains based on scene lux level and lens focus distance. These gains compensate for both color lens shading (i.e., the deviation between R, G, and B channels) and vignetting (i.e., the drop off in pixel intensity around the edges of an image). As scene illuminance increases, the sensor captures more signal from the actual scene, and the lens shading effects begin to appear. To deal with the situation, the lens shading gains are configured to adaptively ‘scale down’ when scene lux approaches zero and ‘scale up’ when scene lux changes from near zero to become larger. The lens shading gain may also be modulated based on the focus distance. For optical systems without zoom, the inventors have discovered that the amount of lens shading fall off changes as focus distance changes.

公开(公告)号：US09418311B2

公开(公告)日：2016-08-16

申请号：US14477679

申请日：2014-09-04

Applicant: Apple Inc.

Inventor： Yingjun Bai ,  Xuemei Zhang

IPC: G06K9/00 ,  G06K9/46 ,  G06F17/30

CPC classification number: G06K9/4661 ,  G06F17/3028 ,  G06K9/46 ,  G06K9/4642 ,  G06K2009/4666 ,  G06T5/007 ,  G06T5/40 ,  G06T2207/20016

Abstract: Methods, devices and computer readable instructions to generate multi-scale tone curves are disclosed. One method includes finding, for a given input image, a global tone curve that exhibits monotonic behavior. The input image may then be partitioned into a first number of sub-regions. For each sub-region, a local tone curve may be determined that has an output level that is constrained to the global tone curve at one or more first luminance levels so that each sub-region's local tone curve's output follows the global tone curve's monotonic behavior. If the resulting local tone curves provide sufficient control of shadow-boost, highlight-suppression, and contrast optimization the first number of local tone curves may be applied directly to the input image. If additional control is needed, each sub-region may again be partitioned and local tone curves determined for each of the new sub-regions.

Abstract translation: 公开了用于产生多尺度色调曲线的方法，装置和计算机可读指令。 一种方法包括为给定输入图像找到呈现单调行为的全局色调曲线。 然后可以将输入图像分割成第一数量的子区域。 对于每个子区域，可以确定本地色调曲线，其具有在一个或多个第一亮度水平下被限制到全局色调曲线的输出电平，使得每个子区域的局部色调曲线的输出遵循全局色调曲线的单调行为 。 如果所得到的本地色调曲线提供了阴影提升，高光抑制和对比度优化的足够控制，则可以将第一数量的局部色调曲线直接应用于输入图像。 如果需要附加控制，则每个子区域可以再次被划分，并且为每个新的子区域确定局部色调曲线。

公开(公告)号：US20160232419A1

公开(公告)日：2016-08-11

申请号：US14614568

申请日：2015-02-05

Applicant: Apple Inc.

Inventor： Yingjun Bai ,  Xuemei Zhang

IPC: G06K9/46 ,  G06K9/52

CPC classification number: G06T5/007 ,  G06T5/40 ,  G06T2207/20016

Abstract: Methods, devices and computer readable instructions to generate region-of-interest (ROI) tone curves are disclosed. One method includes obtaining a statistic for an entire image such as, for example, a luminance statistic. The same statistic may then be found for a specified ROI of the image. A weighted combination of the statistic of the entire image and the statistic of the ROI yields a combined statistic which may then be converted to a ROI-biased tone curve. The weight used to combine the two statistics may be selected to emphasize or de-emphasize the role of the ROI's statistic in the final tone curve.

Abstract translation: 公开了用于生成感兴趣区（ROI）色调曲线的方法，装置和计算机可读指令。 一种方法包括获得整个图像的统计量，例如亮度统计量。 然后可以为图像的指定ROI找到相同的统计量。 整个图像的统计量和ROI的统计量的加权组合产生组合的统计量，然后可以将它们转换为ROI偏置的色调曲线。 可以选择用于组合两个统计量的权重来强调或不强调ROI的统计在最终色调曲线中的作用。