公开(公告)号：WO2016054182A1

公开(公告)日：2016-04-07

申请号：PCT/US2015/053182

申请日：2015-09-30

Applicant: APPLE INC.

Inventor： BEYSSERIE, Sebastien, X. ,  KLIVINGTON, Jason ,  TOFT, Rolf ,  DOEPKE, Frank

IPC: H04N5/14 ,  H04N5/783 ,  G11B15/18 ,  H04N5/915

CPC classification number: H04N5/91 ,  G11B15/1875 ,  H04N5/147 ,  H04N5/77 ,  H04N5/783 ,  H04N5/915

Abstract: Traditionally, time-lapse videos are constructed from images captured at time intervals called "temporal points of interests" or "temporal POIs." Disclosed herein are systems and methods of constructing improved, motion-stabilized time-lapse videos using temporal points of interest and image similarity comparisons. According to some embodiments, a "burst" of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing an image similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest. Selecting the image from a given burst that is most similar to the previous selected image, while minimizing the amount of motion with the previous selected image, allows the system to improve the quality of the resultant time-lapse video by discarding "outlier" or other undesirable images captured in the burst sequence and motion stabilizing the selected image.

Abstract translation: 传统上，延时视频是从被称为“时间兴趣点”或“时间POI”的时间间隔拍摄的图像构建的。 本文公开了使用时间兴趣点和图像相似性比较来构建改进的，运动稳定的延时视频的系统和方法。 根据一些实施例，可以以上述时间感兴趣点为中心捕获图像的“突发”。 然后，可以例如通过在突发序列中的每个图像和在先前的时间点选择的图像之间执行图像相似性比较来分析图像的每个突发序列。 选择与先前选择的图像最相似的给定脉冲串中的图像，同时使用先前选择的图像最小化运动量，允许系统通过丢弃“异常值”或其他信号来提高合成的延时视频的质量 在突发序列中捕获的不期望的图像和稳定所选择的图像的运动。

公开(公告)号：WO2015184408A1

公开(公告)日：2015-12-03

申请号：PCT/US2015/033411

申请日：2015-05-29

Applicant: APPLE INC.

Inventor： MOLGAARD, Claus ,  TICO, Marius ,  TOFT, Rolf ,  HUBEL, Paul, M.

IPC: G06T5/50 ,  H04N5/235 ,  G06T7/20 ,  H04N5/355 ,  H04N5/232

CPC classification number: H04N5/23277 ,  G06T5/002 ,  G06T5/003 ,  G06T5/50 ,  G06T7/20 ,  G06T7/254 ,  G06T11/60 ,  G06T2207/10004 ,  G06T2207/10144 ,  G06T2207/20221 ,  H04N5/23229 ,  H04N5/23254 ,  H04N5/23267 ,  H04N5/2355 ,  H04N5/2356 ,  H04N5/35581 ,  H04N5/3597 ,  H04N5/91

Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images [e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a "spatial difference map." Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

Abstract translation: 公开了从稳定的图像捕获装置捕捉和融合场景的短曝光和长曝光图像的技术。 更具体地，所公开的技术不仅使用共拍摄的短曝光图像和长曝光图像之间的单独的像素差异，而且使用长曝光图像中的遮挡区域的空间结构[例如，长曝光图像的区域 ）呈现由于场景物体运动造成的模糊）。 用于表示长曝光图像的该特征的新型装置是“空间差异图”。 可以使用空间差异图来识别用于融合的短曝光和长曝光图像中的像素，并且在一个实施例中，可以使用空间差异图来识别来自短曝光图像的像素以过滤融合后的图像，以便减少 输出图像中的视觉不连续性。

公开(公告)号：WO2014197154A1

公开(公告)日：2014-12-11

申请号：PCT/US2014/036879

申请日：2014-05-06

Applicant: APPLE INC.

Inventor： TICO, Marius ,  ZHOU, Jianping ,  NARIANI, Schulze, Anita ,  TOFT, Rolf ,  HUBEL, Paul ,  SUN, Wei

IPC: G06T5/00 ,  G06T5/10

CPC classification number: G06K9/46 ,  G06K9/6201 ,  G06T5/003 ,  G06T5/10 ,  G06T2207/10016 ,  G06T2207/10144 ,  G06T2207/20016 ,  G06T2207/20201 ,  G06T2207/20208 ,  G06T2207/20221 ,  H04N5/23267

Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non- reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.

Abstract translation: 描述了用于改善图像稳定操作的系统，方法和计算机可读介质。 公开了一组用于将非参考图像与一组常规拍摄图像中的预选参考帧融合的新方法。 融合方法可以通过使用重影/非重像素的加权平均值来使用软转换，以避免邻域和几乎相似像素之间的突然过渡。 此外，可以基于一组相邻像素而不是针对每个像素独立地进行重影/非重影决定。 替代方法可以包括在每个级别上使用时间融合，时空融合或其组合执行所有捕获图像的多分辨率分解，并组合不同的级别以生成输出图像。

公开(公告)号：WO2013184379A1

公开(公告)日：2013-12-12

申请号：PCT/US2013/042427

申请日：2013-05-23

Applicant: APPLE INC.

Inventor： ZHOU, Jianping ,  BEYSSERIE, Sebastien, X. ,  WILLIAMS, George, E. ,  TOFT, Rolf

IPC: H04N5/232

CPC classification number: H04N5/23248

Abstract: An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute instantaneous motion of the camera for a current frame of the video sequence and accumulated motion of the camera corresponding to motion of a plurality of frames of the video sequence. The controller may compare the instantaneous motion to a first threshold value, compare the accumulated motion to a second threshold value, and set a video stabilization strength parameter for the current frame based on the results of the comparison. A video stabilization unit may perform video stabilization on the current frame according to the frame's strength parameter.

Abstract translation: 一种用于基于运动传感器的视频稳定的装置，方法和计算机可读介质。 运动传感器可以捕获视频序列的运动数据。 控制器可以计算摄像机对视频序列的当前帧的瞬时运动以及对应于视频序列的多个帧的运动的照相机的累积运动。 控制器可以将瞬时运动与第一阈值进行比较，将累积运动与第二阈值进行比较，并且基于比较结果为当前帧设置视频稳定强度参数。 视频稳定单元可以根据帧的强度参数在当前帧上执行视频稳定。