Computer-generated reality recorder

    公开(公告)号:US12299980B2

    公开(公告)日:2025-05-13

    申请号:US18380622

    申请日:2023-10-16

    Applicant: Apple Inc.

    Abstract: Implementations of the subject technology provides analyzing a recording of content. The subject technology generates metadata information based at least in part on the analyzing. The subject technology identifies, based at least in part on at least one of a user preference or a detected event, a region of interest or an object of interest in the recording of content. Based at least in part on the identified region of interest or object of interest, the subject technology generates a modified version of the recording of content. Further, the subject technology stores the modified version of the recording of content for subsequent playback on an electronic device.

    DISTRIBUTED PROCESSING IN COMPUTER GENERATED REALITY SYSTEM

    公开(公告)号:US20240112391A1

    公开(公告)日:2024-04-04

    申请号:US18484236

    申请日:2023-10-10

    Applicant: Apple Inc.

    CPC classification number: G06T15/005 G06F3/147 H04L67/59 G06T2200/16

    Abstract: Techniques are disclosed relating to display devices. In some embodiments, a display device includes a display system configured to display three-dimensional content to a user. The display device is configured to discover, via a network interface, one or more compute nodes operable to facilitate rendering the three-dimensional content and receive information identifying abilities of the one or more compute nodes to facilitate the rendering. Based on the received information, the display device evaluates a set of tasks to identify one or more of the tasks to offload to the one or more compute nodes for facilitating the rendering and distributes, via the network interface, the identified one or more tasks to the one or more compute nodes for processing by the one or more compute nodes.

    Low-Latency Video Matting
    34.
    发明公开

    公开(公告)号:US20240104686A1

    公开(公告)日:2024-03-28

    申请号:US18469984

    申请日:2023-09-19

    Applicant: Apple Inc.

    CPC classification number: G06T1/20 G06T3/40 G06T7/11 G06T2207/20081

    Abstract: Techniques are disclosed herein for implementing a novel, low latency, guidance map-free video matting system, e.g., for use in extended reality (XR) platforms. The techniques may be designed to work with low resolution auxiliary inputs (e.g., binary segmentation masks) and to generate alpha mattes (e.g., alpha mattes configured to segment out any object(s) of interest, such as human hands, from a captured image) in near real-time and in a computationally efficient manner. Further, in a domain-specific setting, the system can function on a captured image stream alone, i.e., it would not require any auxiliary inputs, thereby reducing computational costs—without compromising on visual quality and user comfort. Once an alpha matte has been generated, various alpha-aware graphical processing operations may be performed on the captured images according to the generated alpha mattes (e.g., background replacement operations, synthetic shallow depth of field (SDOF) rendering operations, and/or various XR environment rendering operations).

    Distributed Encoding
    35.
    发明公开

    公开(公告)号:US20230362226A1

    公开(公告)日:2023-11-09

    申请号:US18335669

    申请日:2023-06-15

    Applicant: Apple Inc.

    CPC classification number: H04L65/70 G06F3/012 G02B27/017 H04L65/80 H04L65/762

    Abstract: Techniques are disclosed relating to encoding recorded content for distribution to other computing devices. In some embodiments, a first computing device creates recorded content for transmission to a second computing device configured to present the recorded content. To encode the recorded content, the first computing device detects, via a network interface of the first computing device, one or more computing nodes available to encode the recorded content in one or more formats supported by the second computing device. The first computing device offloads the recorded content via the network interface to the one or more computing nodes for encoding in the one or more formats. In some embodiments, the second computing device receives a request from a user to stream content recorded by a first computing device and requests the content in a first format being encoded by a computing node assisting the first computing device.

    Efficient delivery of multi-camera interactive content

    公开(公告)号:US11533351B2

    公开(公告)日:2022-12-20

    申请号:US17320199

    申请日:2021-05-13

    Applicant: Apple Inc.

    Abstract: Techniques are disclosed relating to encoding recorded content for distribution to other computing devices. In various embodiments, a first computing device records content of a physical environment in which the first computing device is located, the content being deliverable to a second computing device configured to present a corresponding environment based on the recorded content and content recorded by one or more additional computing devices. The first computing device determines a location of the first computing device within the physical environment and encodes the location in a manifest usable to stream the content recorded by the first computing device to the second computing device. The encoded location is usable by the second computing device to determine whether to stream the content recorded by the first computing device.

    Extended reality recorder
    37.
    发明授权

    公开(公告)号:US11521359B2

    公开(公告)日:2022-12-06

    申请号:US17184585

    申请日:2021-02-24

    Applicant: Apple Inc.

    Abstract: Implementations of the subject technology provide systems and methods for recording an extended reality experience in a way that allows the experience to be played back at a later time from a different viewpoint or perspective. This allows computer-generated content that was rendered for display to a user during the recording, to be re-rendered during playback at the correct time and location in the recording, but from a different perspective. In order to facilitate this type of viewer-centric playback, the recording includes a computer-generated content track that references resources for re-rendering the computer-generated content at each point in time in the recording.

    MEDIA COMPOSITOR FOR COMPUTER-GENERATED REALITY

    公开(公告)号:US20220207842A1

    公开(公告)日:2022-06-30

    申请号:US17693881

    申请日:2022-03-14

    Applicant: Apple Inc.

    Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

    Gaze-driven recording of video
    39.
    发明授权

    公开(公告)号:US11350113B2

    公开(公告)日:2022-05-31

    申请号:US17176677

    申请日:2021-02-16

    Applicant: Apple Inc.

    Abstract: Systems and methods for gaze-driven recording of video are described. Some implementations may include accessing gaze data captured using one or more gaze-tracking sensors; applying a temporal filter to the gaze data to obtain a smoothed gaze estimate; determining a region of interest based on the smoothed gaze estimate, wherein the region of interest identifies a subset of a field of view; accessing a frame of video; recording a portion of the frame associated with the region of interest as an enhanced frame of video, wherein the portion of the frame corresponds to a smaller field of view than the frame; and storing, transmitting, or displaying the enhanced frame of video.

    Efficient Delivery of Multi-Camera Interactive Content

    公开(公告)号:US20220094732A1

    公开(公告)日:2022-03-24

    申请号:US17320199

    申请日:2021-05-13

    Applicant: Apple Inc.

    Abstract: Techniques are disclosed relating to encoding recorded content for distribution to other computing devices. In various embodiments, a first computing device records content of a physical environment in which the first computing device is located, the content being deliverable to a second computing device configured to present a corresponding environment based on the recorded content and content recorded by one or more additional computing devices. The first computing device determines a location of the first computing device within the physical environment and encodes the location in a manifest usable to stream the content recorded by the first computing device to the second computing device. The encoded location is usable by the second computing device to determine whether to stream the content recorded by the first computing device.

Patent Agency Ranking