POSITIONAL TRACKING WITHIN AN OBJECT-BASED COORDINATE SYSTEM

    公开(公告)号:US20240378827A1

    公开(公告)日:2024-11-14

    申请号:US18765506

    申请日:2024-07-08

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that track a position of a device within an object-based coordinate system. An example process may include, at a device and prior to movement of an object, acquiring first images of the object, identifying three-dimensional (3D) keypoints on surfaces of the object, and tracking positions of the device in an object-based coordinate system during acquisition of the first images. The process may further include, subsequent to a movement of the object, acquiring second images of the object, identifying the 3D keypoints on surfaces of the object, and tracking positions of the device in the object-based coordinate system during acquisition of the second images based on identifying the 3D keypoints. The process may further include generating a 3D model of the object based on the first and second images and the tracked positions of the device during acquisition of the images.

    Single-pass object scanning
    12.
    发明授权

    公开(公告)号:US11580692B2

    公开(公告)日:2023-02-14

    申请号:US17171434

    申请日:2021-02-09

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that generates a three-dimensional (3D) model based on a selected subset of the images and depth data corresponding to each of the images of the subset. For example, an example process may include acquiring sensor data during movement of the device in a physical environment including an object, the sensor data including images of a physical environment captured via a camera on the device, selecting a subset of the images based on assessing the images with respect to motion-based defects based on device motion and depth data, and generating a 3D model of the object based on the selected subset of the images and depth data corresponding to each of the images of the selected subset.

    Display system having sensors
    13.
    发明授权

    公开(公告)号:US11217021B2

    公开(公告)日:2022-01-04

    申请号:US16361110

    申请日:2019-03-21

    Applicant: Apple Inc.

    Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

    Panoramic light field capture, processing, and display

    公开(公告)号:US11044398B2

    公开(公告)日:2021-06-22

    申请号:US16583176

    申请日:2019-09-25

    Applicant: Apple Inc.

    Abstract: A light field panorama system in which a user holding a mobile device performs a gesture to capture images of a scene from different positions. Additional information, for example position and orientation information, may also be captured. The images and information may be processed to determine metadata including the relative positions of the images and depth information for the images. The images and metadata may be stored as a light field panorama. The light field panorama may be processed by a rendering engine to render different 3D views of the scene to allow a viewer to explore the scene from different positions and angles with six degrees of freedom. Using a rendering and viewing system such as a mobile device or head-mounted display, the viewer may see behind or over objects in the scene, zoom in or out on the scene, or view different parts of the scene.

    TECHNIQUES FOR DISPLAYING AND CAPTURING IMAGES

    公开(公告)号:US20250071248A1

    公开(公告)日:2025-02-27

    申请号:US18948235

    申请日:2024-11-14

    Applicant: Apple Inc.

    Abstract: Embodiments disclosed herein are directed to devices, systems, and methods for separately processing an image stream to generate a first set of images for display by a set of displays and a second set of images for storage or transfer as part of a media capture event. Specifically, a first set of images from an image stream may be used to generate a first set of transformed images, and these transformed images may be displayed on a set of displays. A second set of images may be selected from the image stream in response to a capture request associated with a media capture event, and the second set of images may be used to generate a second set of transformed images. A set of output images may be generated from the second set of transformed images, and the set of output images may be stored or transmitted for later viewing.

    TECHNIQUES FOR DISPLAYING AND CAPTURING IMAGES

    公开(公告)号:US20240406365A1

    公开(公告)日:2024-12-05

    申请号:US18661336

    申请日:2024-05-10

    Applicant: Apple Inc.

    Abstract: Embodiments disclosed herein are directed to devices, systems, and methods for separately processing an image stream to generate a first set of images for display by a set of displays and a second set of images for storage or transfer as part of a media capture event. Specifically, a first set of images from an image stream may be used to generate a first set of transformed images, and these transformed images may be displayed on a set of displays. A second set of images may be selected from the image stream in response to a capture request associated with a media capture event, and the second set of images may be used to generate a second set of transformed images. A set of output images may be generated from the second set of transformed images, and the set of output images may be stored or transmitted for later viewing.

Patent Agency Ranking