公开(公告)号：US20240201512A1

公开(公告)日：2024-06-20

申请号：US18067604

申请日：2022-12-16

Applicant: Apple Inc.

Inventor： Felipe Bacim De Araujo E Silva ,  Noah D. Bedard ,  Bosheng Zhang ,  Brett D. Miller ,  Seung Wook Kim

IPC: G02B30/27 ,  G02B27/01 ,  G06F3/01 ,  G06T15/00 ,  H04N13/275

CPC classification number: G02B30/27 ,  G02B27/017 ,  G06F3/011 ,  G06T15/005 ,  H04N13/275

Abstract: Methods and apparatus for generating images to be displayed on lenticular displays. In these methods, a fixed mesh is generated offline, and in real-time texture information is mapped to the fixed mesh. In an offline process, texture and 3D mesh information for an object is used to render UV map views for multiple viewpoints of the object, view maps are generated from display calibration data, and a lenticular to UV map is generated from the UV map views and view maps. In real-time, texture information is captured, and a composite process is performed that generates a lenticular image for multiple viewpoints by sampling pixels from the texture based on the lenticular to UV map. The lenticular image is then displayed on the lenticular display. Detected positions of persons in the environment may be used to limit the number of viewpoints that are generated during the real-time composite process.

公开(公告)号：US11860368B2

公开(公告)日：2024-01-02

申请号：US17933060

申请日：2022-09-16

Applicant: Apple Inc.

Inventor： Noah D. Bedard ,  Ricardo J. Motta ,  Emanuele Mandelli ,  Daniel R. Hennigan ,  Kathrin Berkner Cieslicki ,  Branko Petljanski ,  Brett D. Miller ,  Kenichi Saito

IPC: G02B27/01 ,  G03B30/00

CPC classification number: G02B27/0172 ,  G03B30/00 ,  G02B2027/014 ,  G02B2027/0123 ,  G02B2027/0129 ,  G02B2027/0138 ,  G02B2027/0198

Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

公开(公告)号：US11720996B2

公开(公告)日：2023-08-08

申请号：US17651001

申请日：2022-02-14

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06V20/10 ,  H04N23/63 ,  H04N23/90 ,  G06T7/50 ,  G06T7/70 ,  H04N5/262 ,  H04N9/73 ,  H04N23/67 ,  G06T7/521 ,  G06T7/593

CPC classification number: G06T3/0093 ,  G06T7/50 ,  G06T7/70 ,  G06V20/10 ,  H04N5/2621 ,  H04N9/73 ,  H04N23/63 ,  H04N23/90 ,  G06T7/521 ,  G06T7/593 ,  G06T2207/20164 ,  G06T2207/30201 ,  H04N23/67

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US10659757B2

公开(公告)日：2020-05-19

申请号：US16241294

申请日：2019-01-07

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Gary L. Vondran, Jr. ,  Manohar B. Srikanth ,  Brett D. Miller

IPC: H04N13/156 ,  H04N7/14 ,  G06F3/01 ,  H04N5/247 ,  H04N5/232 ,  H04N7/18 ,  H04N13/122 ,  H04N13/117 ,  H04N13/243

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

公开(公告)号：US10178371B2

公开(公告)日：2019-01-08

申请号：US15588228

申请日：2017-05-05

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Gary L. Vondran, Jr. ,  Manohar B. Srikanth ,  Brett D. Miller

IPC: H04N7/15 ,  H04N13/156 ,  H04N7/14 ,  G06F3/01 ,  H04N5/247 ,  H04N5/232 ,  H04N13/117 ,  H04N13/243 ,  H04N7/18 ,  H04N13/122

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

公开(公告)号：US20180082482A1

公开(公告)日：2018-03-22

申请号：US15711992

申请日：2017-09-21

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Brett D. Miller ,  Tobias Rick ,  Manohar B. Srikanth

IPC: G06T19/00 ,  H04N13/04 ,  G02B27/01

CPC classification number: G06T19/006 ,  G02B27/0093 ,  G02B27/0172 ,  G02B2027/0136 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0187 ,  G06F1/163 ,  G06F3/011 ,  G06F3/012 ,  G06F3/013 ,  G06F3/015 ,  G06F3/017 ,  H04N13/344 ,  H04N13/383

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.