公开(公告)号：US20170280059A1

公开(公告)日：2017-09-28

申请号：US15457377

申请日：2017-03-13

Applicant: Apple Inc.

Inventor： Damien J. Thivent ,  George E. Williams ,  Jianping Zhou ,  Richard L. Baer ,  Rolf Toft ,  Sebastien X. Beysserie

IPC: H04N5/232 ,  G06T5/20

CPC classification number: H04N5/23287 ,  G06T5/20 ,  H04N5/2254 ,  H04N5/23254 ,  H04N5/23258 ,  H04N5/23267 ,  H04N5/2328

Abstract: Techniques to improve a digital image capture device's ability to stabilize a video stream are presented. According to some embodiments, improved stabilization of captured video frames is provided by intelligently harnessing the complementary effects of both optical image stabilization (OIS) and electronic image stabilization (EIS). In particular, OIS may be used to remove intra-frame motion blur that is typically lower in amplitude and dominates with longer integration times, while EIS may be used to remove residual unwanted frame-to-frame motion that is typically larger in amplitude. The techniques disclosed herein may also leverage information provided from the image capture device's OIS system to perform improved motion blur-aware video stabilization strength modulation, which permits better video stabilization performance in low light conditions, where integration times tend to be longer, thus leading to a greater amount of motion blurring in the output stabilized video.

公开(公告)号：US20170070731A1

公开(公告)日：2017-03-09

申请号：US15256526

申请日：2016-09-03

Applicant: Apple Inc.

Inventor： Benjamin A. Darling ,  Thomas E. Bishop ,  Kevin A. Gross ,  Paul M. Hubel ,  Todd S. Sachs ,  Guangzhi Cao ,  Alexander Lindskog ,  Stefan Weber ,  Jianping Zhou

IPC: H04N17/00 ,  G06T7/00

CPC classification number: H04N17/002 ,  G06T7/337 ,  G06T7/80 ,  G06T7/85

Abstract: Camera calibration includes capturing a first image of an object by a first camera, determining spatial parameters between the first camera and the object using the first image, obtaining a first estimate for an optical center, iteratively calculating a best set of optical characteristics and test setup parameters based on the first estimate for the optical center until the difference in a most recent calculated set of optical characteristics and previously calculated set of optical characteristics satisfies a predetermined threshold, and calibrating the first camera based on the best set of optical characteristics. Multi-camera system calibration may include calibrating, based on a detected misalignment of features in multiple images, the multi-camera system using a context of the multi-camera system and one or more prior stored contexts.

Abstract translation: 相机校准包括通过第一相机捕获对象的第一图像，使用第一图像确定第一相机和对象之间的空间参数，获得光学中心的第一估计值，迭代地计算最佳的一组光学特性和测试设置 基于光学中心的第一估计的参数，直到最近计算出的光学特性集合的差异和先前计算出的光学特性的组合满足预定阈值，并且基于最佳的光学特性集来校准第一相机。 多摄像机系统校准可以包括基于检测到的多个图像中的特征的不对准，使用多摄像机系统的上下文和一个或多个先前存储的上下文来校准多摄像机系统。

公开(公告)号：US09384552B2

公开(公告)日：2016-07-05

申请号：US13911793

申请日：2013-06-06

Applicant: Apple Inc.

Inventor： Jianping Zhou ,  Paul Hubel ,  Marius Tico ,  Anita Nariani Schulze ,  Rolf Toft

IPC: G06T7/00

CPC classification number: G06T7/0028 ,  G06T7/33 ,  G06T2207/10016 ,  G06T2207/20016

Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.

Abstract translation: 描述了用于改善图像稳定操作的系统，方法和计算机可读介质。 公开了一种将非参考图像基于像素的注册到一组共同拍摄的图像中的参考帧的新颖方法，其利用金字塔分解来更有效地检测角落。 所公开的基于像素的注册操作还可以与基于运动传感器数据的注册方法组合以便相对于参考帧来注册非参考图像。 当登记的非参考图像与预先选择的参考图像组合时，所得图像是质量稳定图像。

公开(公告)号：US20160006935A1

公开(公告)日：2016-01-07

申请号：US14467998

申请日：2014-08-25

Applicant: Apple Inc.

Inventor： Jianping Zhou ,  Sebastien X. Beysserie

IPC: H04N5/232

CPC classification number: H04N5/23254 ,  H04N5/23258 ,  H04N5/23267

Abstract: Techniques to improve a digital image capture device's ability to stabilize a video stream in low light capture conditions are presented. In general, techniques are disclosed for stabilizing video images using an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. More particularly, techniques are disclosed for buffering an initial number of input frames so that a “current” frame can use time stamped image capture device motion data from both “past” and “future” frames to adjust the strength of a stabilization metric value so as to keep the current frame within its overscan. The calculated adjustments to the strength of the stabilization metric value for a particular frame may be limited to a target max strength value that is determined based, at least in part, on an estimated amount of motion blur in the current frame.

Abstract translation: 介绍了提高数字图像采集设备在低光捕获条件下稳定视频流的能力的技术。 通常，公开了用于使用过扫描区域稳定视频图像的技术，以及通过在生成第一稳定视频输出帧之前缓冲多个视频输入帧而启用的先行技术。 更具体地，公开了用于缓冲​​初始数量的输入帧的技术，使得“当前”帧可以使用来自“过去”和“未来”帧的时间戳图像捕获设备运动数据来调整稳定度量值的强度 以将当前帧保持在其过扫描内。 对特定帧的稳定度量值的强度的计算的调整可以被限制为至少部分地基于当前帧中的估计的运动模糊量确定的目标最大强度值。

公开(公告)号：US20150350548A1

公开(公告)日：2015-12-03

申请号：US14467987

申请日：2014-08-25

Applicant: Apple Inc.

Inventor： Sebastien X. Beysserie ,  Damien J. Thivent ,  Jianping Zhou ,  Rudolph van der Merwe ,  Jason Klivington ,  Xiaoxing Li ,  Anders Holtsberg ,  Martin Lindberg ,  George E. Williams

IPC: H04N5/232 ,  G06T7/00 ,  G06T5/00

CPC classification number: H04N5/23267 ,  G06T5/003 ,  G06T2207/10016 ,  G06T2207/20201 ,  H04N5/23254

Abstract: Techniques to permit a digital image capture device to stabilize a video stream in real-time (during video capture operations) are presented. In general, techniques are disclosed for stabilizing video images using an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. (Capturing a larger image than is displayed creates a buffer of pixels around the edge of an image; overscan is the term given to this buffer of pixels.) More particularly, techniques are disclosed for buffering an initial number of input frames so that a “current” frame can use motion data from both “past” and “future” frames to adjust the strength of a stabilization metric value so as to keep the current frame within its overscan. This look-ahead and look-behind capability permits a smoother stabilizing regime with fewer abrupt adjustments.

Abstract translation: 呈现允许数字图像捕获设备实时（视频捕获操作期间）稳定视频流的技术。 通常，公开了用于使用过扫描区域稳定视频图像的技术，以及通过在生成第一稳定视频输出帧之前缓冲多个视频输入帧而启用的先行技术。 （捕获比所显示的图像更大的图像在图像的边缘周围产生像素的缓冲器;过扫描是给予该像素缓冲器的术语。）更具体地，公开了用于缓冲​​初始数量的输入帧的技术， 当前“帧可以使用来自”过去“和”未来“帧的运动数据来调整稳定度量值的强度，以便将当前帧保持在其过扫描内。 这种前瞻性和后仰能力允许更平稳的稳定方案，具有较少的突然调整。