公开(公告)号：WO2011049777A1

公开(公告)日：2011-04-28

申请号：PCT/US2010/052264

申请日：2010-10-12

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N5/217 ,  H04N5/357 ,  H04N5/359 ,  H04N5/367 ,  H04N9/04

CPC classification number: H04N5/3572 ,  H04N5/2176 ,  H04N5/23229 ,  H04N5/359 ,  H04N5/3675 ,  H04N9/045

Abstract: Various techniques are provided for the detection and correction of defective pixels in an image sensor 90. In accordance with one embodiment, a static defect table storing the locations of known static defects is provided, and the location of a current pixel is compared to the static defect table. If the location of the current pixel is found in the static defect table, the current pixel is identified as a static defect and is corrected using the value of the previous pixel of the same color. If the current pixel is not identified as a static defect, a dynamic defect detection process 444 includes comparing pixel-to-pixel gradients between the current pixel a set of neighboring pixels against a dynamic defect threshold. If a dynamic defect is detected, a replacement value for correcting the dynamic defect may be determined by interpolating the value of two neighboring pixels on opposite sides of the current pixel in a direction exhibiting the smallest gradient.

Abstract translation: 提供了用于图像传感器90中的缺陷像素的检测和校正的各种技术。根据一个实施例，提供存储已知静态缺陷的位置的静态缺陷表，并将当前像素的位置与静态 缺陷表。 如果在静态缺陷表中找到当前像素的位置，则将当前像素识别为静态缺陷，并使用相同颜色的先前像素的值进行校正。 如果当前像素未被识别为静态缺陷，则动态缺陷检测处理444包括将当前像素与一组相邻像素之间的像素与像素梯度相比较的动态缺陷阈值。 如果检测到动态缺陷，则可以通过在呈现最小梯度的方向上内插当前像素的相对侧上的两个相邻像素的值来确定用于校正动态缺陷的替换值。

公开(公告)号：WO2012030617A1

公开(公告)日：2012-03-08

申请号：PCT/US2011/049121

申请日：2011-08-25

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N5/232

CPC classification number: H04N5/23212 ,  G03B3/10 ,  G03B13/36

Abstract: Techniques are provided for determining an optimal focal position using auto-focus statistics. In one embodiment, such techniques may include generating coarse and fine auto-focus scores for determining an optimal focal length at which to position a lens 88 associated with the image sensor 90. For instance, the statistics logic 680 may determine a coarse position that indicates an optimal focus area which, in one embodiment, may be determined by searching for the first coarse position in which a coarse auto-focus score decreases with respect to a coarse auto-focus score at a previous position. Using this position as a starting point for fine score searching, the optimal focal position may be determined by searching for a peak in fine auto-focus scores. In another embodiment, auto-focus statistics may also be determined based on each color of the Bayer RGB, such that, even in the presence of chromatic aberrations, relative auto-focus scores for each color may be used to determine the direction of focus.

Abstract translation: 提供了使用自动对焦统计来确定最佳焦点位置的技术。 在一个实施例中，这样的技术可以包括产生用于确定与图像传感器90相关联的透镜88的最佳焦距的粗略和精细自动对焦分数。例如，统计逻辑680可以确定指示 在一个实施例中，可以通过搜索粗略的自动对焦得分相对于在先前位置处的粗略自动对焦得分而减小的第一粗略位置来确定最佳聚焦区域。 使用该位置作为精细分数搜索的起点，可以通过在精细自动对焦分数中搜索峰值来确定最佳焦点位置。 在另一个实施例中，还可以基于拜耳RGB的每种颜色来确定自动对焦统计，使得即使在存在色差的情况下，也可以使用每种颜色的相对自动对焦分数来确定焦点的方向。

公开(公告)号：WO2011049781A1

公开(公告)日：2011-04-28

申请号：PCT/US2010/052293

申请日：2010-10-12

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N9/04 ,  G06T3/40

CPC classification number: H04N9/045 ,  G06T3/4015 ,  H04N5/3675 ,  H04N2209/046

Abstract: Various techniques are provided herein for the demosaicing of images acquired and processed by an imaging system. The imaging system includes an image signal processor 32 and image sensors 30 utilizing color filter arrays (CFA) for acquiring red, green, and blue color data using one pixel array. In one embodiment, the CFA may include a Bayer pattern. During image signal processing, demosaicing may be applied to interpolate missing color samples from the raw image pattern. In one embodiment, interpolation for the green color channel may include employing edge-adaptive filters with weighted gradients of horizontal and vertical filtered values. The red and blue color channels may be interpolated using color difference samples with co-located interpolated values of the green color channel. In another embodiment, interpolation of the red and blue color channels may be performed using color ratios (e.g., versus color difference data).

Abstract translation: 本文提供了用于对由成像系统获取和处理的图像进行去马赛克的各种技术。 成像系统包括图像信号处理器32和利用彩色滤光片阵列（CFA）的图像传感器30，用于使用一个像素阵列获取红，绿和蓝色数据。 在一个实施例中，CFA可以包括拜耳模式。 在图像信号处理期间，可以应用去马赛克来从原始图像图案插值丢失的颜色样本。 在一个实施例中，绿色通道的插值可以包括采用具有水平和垂直滤波值的加权梯度的边缘自适应滤波器。 可以使用具有绿色通道的相同位置的内插值的色差样本来内插红色和蓝色色彩通道。 在另一个实施例中，可以使用颜色比（例如，与色差数据）来执行红色和蓝色色彩通道的插值。

公开(公告)号：WO2014093510A1

公开(公告)日：2014-06-19

申请号：PCT/US2013/074432

申请日：2013-12-11

Applicant: APPLE INC.

Inventor： RIDENOUR, Robert L. ,  FREDERIKSEN, Jeffrey E. ,  HENDRY, Ian C.

IPC: H04N19/37 ,  H04N19/137 ,  H04N19/176 ,  H04N19/127 ,  H04N19/115 ,  H04N19/12 ,  H04N19/63 ,  H04N19/46 ,  H04N19/30 ,  H04N19/107 ,  H04N19/15 ,  H04N19/547

CPC classification number: H04N19/137 ,  H04N19/107 ,  H04N19/115 ,  H04N19/12 ,  H04N19/15 ,  H04N19/176 ,  H04N19/30 ,  H04N19/46 ,  H04N19/547 ,  H04N19/63

Abstract: Lossless image compression using differential transfers may involve an image compression unit receiving image data for an image in a sequence of images and transmitting the image data such that image data for at least some image tiles is transmitted using lossy compression due to resource limitations. The image compression unit may then receive image data for a subsequent image in the sequence and determine that the image data for at least some tiles does not change relative to the image data for corresponding tiles of the previous image. The image compression unit may then transmit image data in a manner sufficient to create lossless versions of tiles for which lossily compressed image data was sent previously.

Abstract translation: 使用差分传送的无损图像压缩可以涉及图像压缩单元，其接收图像序列中的图像的图像数据，并且传输图像数据，使得由于资源限制，使用有损压缩来传送至少一些图像块的图像数据。 然后，图像压缩单元可以接收序列中的后续图像的图像数据，并且确定用于至少一些瓦片的图像数据相对于先前图像的相应瓦片的图像数据不会改变。 然后，图像压缩单元可以以足以创建先前发送有损压缩图像数据的瓦片的无损版本的方式来发送图像数据。

公开(公告)号：WO2012050677A1

公开(公告)日：2012-04-19

申请号：PCT/US2011/050056

申请日：2011-08-31

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N5/225 ,  H04N5/235

CPC classification number: H04N5/2256 ,  H04N5/225 ,  H04N5/228 ,  H04N5/2354

Abstract: Certain aspects of this disclosure relate to an image signal processing system 32 that includes a flash controller 550 that is configured to activate a flash device prior to the start of a target image frame by using a sensor timing signal. In one embodiment, the flash controller 550 receives a delayed sensor timing signal and determines a flash activation start time by using the delayed sensor timing signal to identify a time corresponding to the end of the previous frame, increasing that time by a vertical blanking time, and then subtracting a first offset to compensate for delay between the sensor timing signal and the delayed sensor timing signal. Then, the flash controller 550 subtracts a second offset to determine the flash activation time, thus ensuring that the flash is activated prior to receiving the first pixel of the target frame.

Abstract translation: 本公开的某些方面涉及图像信号处理系统32，图像信号处理系统32包括被配置为通过使用传感器定时信号在目标图像帧开始之前激活闪光装置的闪光灯控制器550。 在一个实施例中，闪光灯控制器550接收延迟的传感器定时信号，并且通过使用延迟的传感器定时信号来确定闪光激活开始时间，以识别对应于先前帧的结束的时间，通过垂直消隐时间增加该时间， 然后减去第一偏移量来补偿传感器定时信号和延迟的传感器定时信号之间的延迟。 然后，闪光灯控制器550减去第二偏移以确定闪光激活时间，从而确保在接收到目标帧的第一像素之前闪光被激活。

公开(公告)号：EP2491710B1

公开(公告)日：2018-07-04

申请号：EP10766203.3

申请日：2010-10-12

Applicant: Apple Inc.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N5/217 ,  H04N5/357 ,  H04N5/359 ,  H04N5/367 ,  H04N9/04

CPC classification number: H04N5/3572 ,  H04N5/2176 ,  H04N5/23229 ,  H04N5/359 ,  H04N5/3675 ,  H04N9/045

Abstract: Various techniques are provided for the detection and correction of defective pixels in an image sensor. In accordance with one embodiment, a static defect table storing the locations of known static defects is provided, and the location of a current pixel is compared to the static defect table. If the location of the current pixel is found in the static defect table, the current pixel is identified as a static defect and is corrected using the value of the previous pixel of the same color. If the current pixel is not identified as a static defect, a dynamic defect detection process includes comparing pixel-to-pixel gradients between the current pixel a set of neighboring pixels against a dynamic defect threshold. If a dynamic defect is detected, a replacement value for correcting the dynamic defect may be determined by interpolating the value of two neighboring pixels on opposite sides of the current pixel in a direction exhibiting the smallest gradient.

公开(公告)号：WO2012047426A1

公开(公告)日：2012-04-12

申请号：PCT/US2011/050029

申请日：2011-08-31

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N21/43 ,  H04N21/2368

CPC classification number: H04N21/4223 ,  H04N21/41407 ,  H04N21/4302

Abstract: The present disclosure provides techniques for performing audio-video synchronization using an image signal processing system. In one embodiment, a time code register provides a current time stamp when sampled. The value of the time code register may be incremented at regular intervals based on a clock of the image signal processing system. At the start of a current frame acquired by an image sensor, the time code register is sampled, and a timestamp is stored into a timestamp register associated with the image sensor. The timestamp is then read from the timestamp register and written to a set of metadata associated with the current frame. The timestamp stored in the frame metadata may then be used to synchronize the current frame with a corresponding set of audio data.

Abstract translation: 本公开提供了使用图像信号处理系统执行音频 - 视频同步的技术。 在一个实施例中，时间码寄存器提供当采样时的当前时间戳。 可以基于图像信号处理系统的时钟以规则的间隔增加时间码寄存器的值。 在由图像传感器获取的当前帧的开始处，时间码寄存器被采样，并且时间戳被存储到与图像传感器相关联的时间戳寄存器中。 然后从时间戳寄存器读取时间戳并写入与当前帧相关联的一组元数据。 存储在帧元数据中的时间戳可以用于将当前帧与对应的一组音频数据同步。

公开(公告)号：WO2011049779A1

公开(公告)日：2011-04-28

申请号：PCT/US2010/052272

申请日：2010-10-12

Applicant: APPLE INC. ,  CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: G06T5/50 ,  H04N5/232 ,  H04N5/14 ,  H04N9/04

CPC classification number: H04N5/232 ,  G06T5/002 ,  G06T5/008 ,  G06T2207/20182 ,  H04N5/23232 ,  H04N5/2355 ,  H04N9/045

Abstract: Various techniques for temporally filtering raw image data acquired by an image sensor are provided. In one embodiment, a temporal filter determines a spatial location of a current pixel and identifies at least one collocated reference pixel from a previous frame. A motion delta value is determined based at least partially upon the current pixel and its collocated reference pixel. Next, an index is determined based upon the motion delta value and a motion history value corresponding to the spatial location of the current pixel, but from the previous frame. Using the index, a first filtering coefficient may be selected from a motion table. After selecting the first filtering coefficient, an attenuation factor may be selected from a luma table based upon the value of the current pixel, and a second filtering coefficient may subsequently be determined based upon the selected attenuation factor and the first filtering coefficient. The temporally filtered output value corresponding to the current pixel may then be calculated based upon the second filtering coefficient, the current pixel, and the collocated reference pixel.

Abstract translation: 提供了用于时间滤波由图像传感器获取的原始图像数据的各种技术。 在一个实施例中，时间滤波器确定当前像素的空间位置并且识别来自先前帧的至少一个并置参考像素。 至少部分地基于当前像素及其并置参考像素确定运动增量值。 接下来，基于与当前像素的空间位置对应的运动增量值和运动历史值，但是从前一帧确定索引。 使用该索引，可以从运动表中选择第一滤波系数。 在选择第一滤波系数之后，可以基于当前像素的值从亮度表中选择衰减因子，并且随后可以基于所选择的衰减因子和第一滤波系数来确定第二滤波系数。 然后可以基于第二滤波系数，当前像素和并置参考像素来计算对应于当前像素的时间滤波的输出值。

公开(公告)号：EP3386197A1

公开(公告)日：2018-10-10

申请号：EP18172698.5

申请日：2013-12-11

Applicant: Apple Inc.

Inventor： RIDENOUR, Robert L. ,  FREDERIKSEN, Jeffrey E. ,  HENDRY, Ian C.

IPC: H04N19/115 ,  H04N19/12 ,  H04N19/137

CPC classification number: H04N19/137 ,  H04N19/107 ,  H04N19/115 ,  H04N19/12 ,  H04N19/15 ,  H04N19/176 ,  H04N19/30 ,  H04N19/46 ,  H04N19/547 ,  H04N19/63

Abstract: Lossless image compression using differential transfers may involve an image compression unit receiving image data for an image in a sequence of images and transmitting the image data such that image data for at least some image tiles is transmitted using lossy compression due to resource limitations. The image compression unit may then receive image data for a subsequent image in the sequence and determine that the image data for at least some tiles does not change relative to the image data for corresponding tiles of the previous image. The image compression unit may then transmit image data in a manner sufficient to create lossless versions of tiles for which lossily compressed image data was sent previously.

公开(公告)号：EP2622867A1

公开(公告)日：2013-08-07

申请号：EP11760905.7

申请日：2011-08-31

Applicant: Apple Inc.

Inventor： CÔTÉ, Guy ,  FREDERIKSEN, Jeffrey E.

IPC: H04N9/73 ,  G06K9/00

CPC classification number: H04N9/735 ,  G06K9/00234 ,  G06T3/4015 ,  G06T5/008 ,  H04N5/23219

Abstract: Disclosed embodiments provide for a an image signal processing system 32 that includes back-end pixel processing unit 120 that receives pixel data after being processed by at least one of a front-end pixel processing unit 80 and a pixel processing pipeline 82. In certain embodiments, the back-end processing unit 120 receives luma/chroma image data and may be configured to apply face detection operations, local tone mapping, bright, contrast, color adjustments, as well as scaling. Further, the back-end processing unit 120 may also include a back-end statistics unit 2208 that may collect frequency statistics. The frequency statistics may be provided to an encoder 118 and may be used to determine quantization parameters that are to be applied to an image frame.

Abstract translation: 公开的实施例提供一种用于图像信号处理系统也包括后端像素处理单元做由一个前端像素处理单元中的至少一个和像素处理流水线处理后接收的像素数据。 在某些实施例中，后端处理单元接收亮度/色度图像数据，并且可以被配置成应用面部检测操作，局部色调映射，亮度，对比度，色彩调整，以及缩放。 此外，后端处理单元可以因此包括后端统计单元也可以收集频率的统计数据。 频率统计可以在编码器被提供到并可能被用于确定性地矿量化参数也将被应用到的图像帧。