公开(公告)号：WO2012158287A1

公开(公告)日：2012-11-22

申请号：PCT/US2012/033010

申请日：2012-04-11

Applicant: APPLE INC. ,  BHOGAL, Nikhil ,  DOEPKE, Frank

Inventor： BHOGAL, Nikhil ,  DOEPKE, Frank

IPC: G06T3/40

CPC classification number: G06T3/4038

Abstract: This disclosure pertains to devices, methods, and computer readable media for performing panoramic photography processing techniques in handheld personal electronic devices. A few generalized steps may be used to carry out the panoramic photography processing techniques described herein: 1.) acquiring image data from the electronic device's image sensor's image stream; 2.) displaying a scaled preview version of the image data in real-time on the device's display; 3.) performing "motion filtering" on the acquired image data; 4.) generating full-resolution and lower-resolution versions of portions of the images that are not filtered out by the "motion filtering" process; 5.) substantially simultaneously "stitching" both the full-resolution and lower-resolution image portions together to create the panoramic scene; and 6.) substantially simultaneously sending the stitched version of the lower-resolution image portions to a preview region on the device's display and storing the stitched version of the full-resolution image portions to a memory.

Abstract translation: 本公开涉及用于在手持个人电子设备中执行全景摄影处理技术的设备，方法和计算机可读介质。 可以使用几个通用步骤来执行本文描述的全景摄影处理技术：1.）从电子设备的图像传感器的图像流中获取图像数据; 2.）在设备的显示屏上实时显示图像数据的缩放预览版本; 对所获取的图像数据执行“运动滤波”; 4.）生成不被“运动过滤”过程滤除的图像部分的全分辨率和低分辨率版本; ）基本上同时将全分辨率和低分辨率图像部分“拼接”在一起以创建全景场景; ）基本上同时将较低分辨率图像部分的缝合版本发送到设备显示器上的预览区域，并将全分辨率图像部分的缝合版本存储到存储器中。

公开(公告)号：WO2008019321A2

公开(公告)日：2008-02-14

申请号：PCT/US2007/075199

申请日：2007-08-03

Applicant: APPLE INC. ,  NEUBRAND, Hans-werner ,  DOEPKE, Frank

Inventor： NEUBRAND, Hans-werner ,  DOEPKE, Frank

IPC: G06F3/048

CPC classification number: G06F3/04847 ,  G06F3/04812 ,  G06F3/0482 ,  G06F3/04855

Abstract: A graphical user interface (GUI) element permits a user to control an application in both a coarse manner and a fine manner. When a cursor is moved to coincide or overlap the displayed GUI element, parameter adjustment is made at a first (coarse) granularity so that rapid changes to the target parameter can be made ( e.g ., displayed zoom level, image rotation or playback volume). As the cursor is moved away from the displayed GUI element, parameter adjustment is made at a second (fine) granularity so that fine changes to the target parameter can be made. In one embodiment, the further the cursor is moved from the displayed GUI element, the finer the control.

Abstract translation: 图形用户界面（GUI）元素允许用户粗略和精细地控制应用程序。 当光标被移动以与所显示的GUI元素重合或重叠时，以第一（粗）粒度进行参数调整，使得可以对目标参数进行快速改变（例如，显示的缩放级别 ，图像旋转或播放音量）。 当光标离开显示的GUI元素时，以第二（精细）粒度进行参数调整，以便可以对目标参数进行精细改变。 在一个实施例中，光标从显示的GUI元素移动得越远，控制越精细。

公开(公告)号：EP2049981A2

公开(公告)日：2009-04-22

申请号：EP07813775.9

申请日：2007-08-03

Applicant: Apple Inc.

Inventor： NEUBRAND, Hans-werner ,  DOEPKE, Frank

IPC: G06F3/048

CPC classification number: G06F3/04847 ,  G06F3/04812 ,  G06F3/0482 ,  G06F3/04855

Abstract: A graphical user interface (GUI) element permits a user to control an application in both a coarse manner and a fine manner. When a cursor is moved to coincide or overlap the displayed GUI element, parameter adjustment is made at a first (coarse) granularity so that rapid changes to the target parameter can be made (e.g., displayed zoom level, image rotation or playback volume). As the cursor is moved away from the displayed GUI element, parameter adjustment is made at a second (fine) granularity so that fine changes to the target parameter can be made. In one embodiment, the further the cursor is moved from the displayed GUI element, the finer the control.

公开(公告)号：EP2601640A1

公开(公告)日：2013-06-12

申请号：EP11738365.3

申请日：2011-07-19

Applicant: Apple Inc.

Inventor： ZIMMER, Mark ,  STAHL, Geoff ,  HAYWARD, David ,  DOEPKE, Frank

IPC: G06T15/20

CPC classification number: G06F3/04815 ,  G06F3/005 ,  G06F3/013 ,  G06F3/017 ,  G06F3/0346 ,  G06F3/0488 ,  G06F2203/0381 ,  G06T15/20

Abstract: The techniques disclosed herein use a compass, MEMS accelerometer, GPS module, and MEMS gyrometer to infer a frame of reference for a hand-held device. This can provide a true Frenet frame, i.e., X- and Y-vectors for the display, and also a Z-vector that points perpendicularly to the display. In fact, with various inertial clues from accelerometer, gyrometer, and other instruments that report their states in real time, it is possible to track the Frenet frame of the device in real time to provide a continuous 3D frame-of-reference. Once this continuous frame of reference is known, the position of a user's eyes may either be inferred or calculated directly by using a device's front-facing camera. With the position of the user's eyes and a continuous 3D frame-of-reference for the display, more realistic virtual 3D depictions of the objects on the device's display may be created and interacted with by the user.

Abstract translation: 的技术的光盘游离缺失使用罗盘，MEMS加速度计，GPS模块，和MEMS陀螺仪推断的参考框架用于一个手持式装置。 这可以提供一个真正的Frenet标帧，即，X和Y向量的显示，因而Z-向量做点垂直于显示器。 事实上，与加速度计，陀螺仪和其他仪器的各种惯性线索做实时报告他们的状态，它可以跟踪设备的Frenet标架实时提供连续3D帧的参考。 一旦参考该连续帧是已知的，用户的眼睛的位置可以被推断任一者或通过使用设备的前置摄像头直接计算。 与用户的眼睛的位置和一个连续的3D帧的参考的显示，在设备的显示器上的对象的更加逼真的虚拟3D描绘可以被创建并与由用户交互。

公开(公告)号：WO2016200741A1

公开(公告)日：2016-12-15

申请号：PCT/US2016/036078

申请日：2016-06-06

Applicant: APPLE INC.

Inventor： MOLGAARD, Claus ,  KEATING, Brett, M. ,  WILLIAMS, George, E. ,  ZULIANI, Marco ,  WONG, Vincent, Y. ,  DOEPKE, Frank ,  TIRA-THOMPSON, Ethan, J.

IPC: H04N5/232 ,  H04N5/262

CPC classification number: G11B27/34 ,  G06K9/00268 ,  G06K9/00718 ,  G06K9/4661 ,  H04N5/23229 ,  H04N5/23245 ,  H04N5/2352 ,  H04N5/2353 ,  H04N5/2621

Abstract: The invention relates to systems, methods, and computer readable media for responding to a user snapshot request by capturing anticipatory pre-snapshot image data as well as post-snapshot image data. The captured information may be used, depending upon the embodiment, to create archival image information and image presentation information that is both useful and pleasing to a user. The captured information may automatically be trimmed or edited to facilitate creating an enhanced image, such as a moving still image. Varying embodiments of the invention offer techniques for trimming and editing based upon the following: exposure, brightness, focus, white balance, detected motion of the camera, substantive image analysis, detected sound, image metadata, and/or any combination of the foregoing.

Abstract translation: 本发明涉及通过捕获预先预先快照图像数据以及快照后图像数据来响应用户快照请求的系统，方法和计算机可读介质。 根据实施例，可以使用捕获的信息来创建对用户有用和令人愉快的档案图像信息和图像呈现信息。 捕获的信息可以自动修剪或编辑，以便于创建增强图像，例如移动的静止图像。 本发明的不同实施例提供了基于以下内容进行修剪和编辑的技术：曝光，亮度，焦点，白平衡，检测到的相机运动，实质图像分析，检测到的声音，图像元数据和/或前述的任何组合。

公开(公告)号：WO2012158307A1

公开(公告)日：2012-11-22

申请号：PCT/US2012/034625

申请日：2012-04-23

Applicant: APPLE INC. ,  DOEPKE, Frank

Inventor： DOEPKE, Frank

IPC: G06T3/40 ,  H04N5/232

CPC classification number: H04N5/23238 ,  G06T3/4038 ,  H04N5/2628

Abstract: This disclosure pertains to devices, methods, and computer readable media for performing positional sensor-assisted panoramic photography techniques in handheld personal electronic devices. Generalized steps that may be used to carry out the panoramic photography techniques described herein include, but are not necessarily limited to: 1.) acquiring image data from the electronic device's image sensor; 2.) performing "motion filtering" on the acquired image data, e.g., using information returned from positional sensors of the electronic device to inform the processing of the image data; 3.) performing image registration between adjacent captured images; 4.) performing geometric corrections on captured image data, e.g., due to perspective changes and/or camera rotation about a non-center of perspective (COP) camera point; and 5.) "stitching" the captured images together to create the panoramic scene, e.g., blending the image data in the overlap area between adjacent captured images. The resultant stitched panoramic image may be cropped before final storage.

Abstract translation: 本公开涉及用于在手持式个人电子设备中执行位置传感器辅助全景摄影技术的设备，方法和计算机可读介质。 可以用于执行本文描述的全景摄影技术的通用步骤包括但不限于：1.从电子设备的图像传感器获取图像数据; 2.对所获取的图像数据执行“运动滤波”，例如使用从电子设备的位置传感器返回的信息来通知图像数据的处理; 3）在相邻拍摄图像之间执行图像配准; 4.）对捕获的图像数据进行几何校正，例如由于透视变化和/或关于非中心透镜（COP）相机点的相机旋转; 和5.）将捕获的图像拼接在一起以创建全景场景，例如，将相邻拍摄图像之间的重叠区域中的图像数据混合。 最终的缝合全景图像可以在最终存储之前被裁剪。

公开(公告)号：WO2011126500A1

公开(公告)日：2011-10-13

申请号：PCT/US2010/049828

申请日：2010-09-22

Applicant: APPLE INC. ,  DOEPKE, Frank

Inventor： DOEPKE, Frank

IPC: G06K9/00 ,  H04N7/14 ,  H04N7/26

CPC classification number: H04N7/147 ,  G06K9/00234 ,  H04N19/115 ,  H04N19/14 ,  H04N19/167 ,  H04N19/176

Abstract: In many videoconferencing applications, bandwidth is at a premium, and thus, it is important to encode a given video frame intelligently. It is often desirable that a larger amount of information be spent encoding the more important parts of the video frame, e.g., human facial features, whereas the less important parts of the video frame can be compressed at higher rates. Thus, there is need for an apparatus, computer readable medium, processor, and method for intelligent skin tone and facial feature aware videoconferencing compression that can "suggest" intelligent macroblock compression ratios to a video encoder. The suggestion of compression rates can be based at least in part on a determination of which macroblocks in a given video frame are likely to contain skin tones, likely to contain features (e.g., edges), likely to contain features in or near skin tone regions, or likely to contain neither skin tones nor features.

Abstract translation: 在许多视频会议应用中，带宽是非常重要的，因此，对于给定的视频帧进行智能编码是很重要的。 通常希望花更大量的信息来编码视频帧的更重要部分，例如人脸特征，而视频帧中不太重要的部分可以以更高的速率被压缩。 因此，需要一种用于智能肤色和面部特征识别视频会议压缩的装置，计算机可读介质，处理器和方法，其能够向视频编码器“建议”智能宏块压缩比。 压缩率的建议可以至少部分地基于确定给定视频帧中哪些宏块可能包含可能包含特征（例如，边缘）的肤色，其可能包含皮肤色调区域或附近的特征 ，或者可能既不含皮肤色调也不含有特征。

公开(公告)号：WO2017039853A1

公开(公告)日：2017-03-09

申请号：PCT/US2016/042816

申请日：2016-07-18

Applicant: APPLE INC.

Inventor： BISHOP, Thomas, E. ,  LINDSKOG, Alexander ,  MOLGAARD, Claus ,  DOEPKE, Frank

IPC: H04N5/235

CPC classification number: H04N13/106 ,  G06T5/002 ,  G06T7/593 ,  H04N5/23229 ,  H04N5/2356

Abstract: Generating an image with a selected level of background blur includes capturing, by a first image capture device, a plurality of frames of a scene, wherein each of the plurality of frames has a different focus depth, obtaining a depth map of the scene, determining a target object and a background in the scene based on the depth map, determining a goal blur for the background, and selecting, for each pixel in an output image, a corresponding pixel from the focus stack.

Abstract translation: 生成具有所选择的背景模糊级别的图像包括由第一图像捕获装置捕获场景的多个帧，其中多个帧中的每一个具有不同的焦点深度，获得场景的深度图，确定 基于深度图的场景中的目标对象和背景，确定背景的目标模糊，以及从输出图像中的每个像素选择来自焦点堆栈的对应像素。

公开(公告)号：WO2016200734A1

公开(公告)日：2016-12-15

申请号：PCT/US2016/036035

申请日：2016-06-06

Applicant: APPLE INC.

Inventor： LINDSKOG, Alexander ,  DOEPKE, Frank ,  BRUNNER, Ralph ,  BISHOP, Thomas, E.

IPC: H04N5/232 ,  H04N5/235

CPC classification number: H04N5/23212 ,  G02B7/36 ,  G02B27/0075 ,  G06T7/50 ,  H04N5/2254 ,  H04N5/23293 ,  H04N5/2356 ,  H04N5/357

Abstract: Generating a focus stack, including receiving initial focus data that identifies a plurality of target depths, positioning a lens at a first position to capture a first image at a first target depth of the plurality of target depths, determining, in response to capturing the first image and prior to capturing additional images, a sharpness metric for the first image, capturing, in response to determining that the sharpness metric for the first image is an unacceptable value, a second image at a second position based on the sharpness metric, wherein the second position is not included in the plurality of target depths, determining that a sharpness metric for the second image is an acceptable value, and generating a focus stack using the second image.

Abstract translation: 产生焦点堆叠，包括接收识别多个目标深度的初始聚焦数据，将透镜定位在第一位置以捕获在多个目标深度的第一目标深度处的第一图像，响应于捕获第一 图像，并且在捕获附加图像之前，针对第一图像的锐度度量，响应于确定第一图像的清晰度度量是不可接受的值，捕获，基于清晰度度量在第二位置处的第二图像，其中 第二位置不包括在多个目标深度中，确定第二图像的锐度度量是可接受的值，并且使用第二图像生成焦点堆叠。

公开(公告)号：WO2016054182A1

公开(公告)日：2016-04-07

申请号：PCT/US2015/053182

申请日：2015-09-30

Applicant: APPLE INC.

Inventor： BEYSSERIE, Sebastien, X. ,  KLIVINGTON, Jason ,  TOFT, Rolf ,  DOEPKE, Frank

IPC: H04N5/14 ,  H04N5/783 ,  G11B15/18 ,  H04N5/915

CPC classification number: H04N5/91 ,  G11B15/1875 ,  H04N5/147 ,  H04N5/77 ,  H04N5/783 ,  H04N5/915

Abstract: Traditionally, time-lapse videos are constructed from images captured at time intervals called "temporal points of interests" or "temporal POIs." Disclosed herein are systems and methods of constructing improved, motion-stabilized time-lapse videos using temporal points of interest and image similarity comparisons. According to some embodiments, a "burst" of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing an image similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest. Selecting the image from a given burst that is most similar to the previous selected image, while minimizing the amount of motion with the previous selected image, allows the system to improve the quality of the resultant time-lapse video by discarding "outlier" or other undesirable images captured in the burst sequence and motion stabilizing the selected image.

Abstract translation: 传统上，延时视频是从被称为“时间兴趣点”或“时间POI”的时间间隔拍摄的图像构建的。 本文公开了使用时间兴趣点和图像相似性比较来构建改进的，运动稳定的延时视频的系统和方法。 根据一些实施例，可以以上述时间感兴趣点为中心捕获图像的“突发”。 然后，可以例如通过在突发序列中的每个图像和在先前的时间点选择的图像之间执行图像相似性比较来分析图像的每个突发序列。 选择与先前选择的图像最相似的给定脉冲串中的图像，同时使用先前选择的图像最小化运动量，允许系统通过丢弃“异常值”或其他信号来提高合成的延时视频的质量 在突发序列中捕获的不期望的图像和稳定所选择的图像的运动。