Immersive media content encoding and rendering

    公开(公告)号:US12299805B1

    公开(公告)日:2025-05-13

    申请号:US17804801

    申请日:2022-05-31

    Applicant: Apple Inc.

    Abstract: A decoding computing device determines a focus area of a view of volumetric visual content, such as a three-dimensional (3D) object or scene, and indicates the focus area in a request for compressed volumetric visual content. A server provides a depth map and an attribute atlas that includes attribute and/or texture information for the focus area that is signaled at a higher resolution than other portions of the volumetric visual content. The decoding device also applies one or more mesh simplification techniques, wherein a higher resolution mesh is used for the focus area. The textures/attributes are projected onto the mesh representations, wherein the mesh representations and projected textures are used to reconstruct the volumetric visual content. Hole filling techniques are then applied, wherein a more sophisticated hole filling technique is used in the focus area.

    Multimodal inputs for computer-generated reality

    公开(公告)号:US12242664B2

    公开(公告)日:2025-03-04

    申请号:US18204892

    申请日:2023-06-01

    Applicant: Apple Inc.

    Abstract: Implementations of the subject technology provide determining an operating mode of an electronic device based at least in part on whether the electronic device is communicatively coupled to an associated base device. Based on the determined operating mode, the subject technology identifies a set of input modalities for initiating a recording of content within a field of view of the electronic device. The subject technology monitors sensor information generated by at least one sensor included in, or communicatively coupled to, the electronic device. Further, the subject technology initiates the recording of content within the field of view of the electronic device when the monitored sensor information indicates that at least one of the identified set of input modalities has been triggered.

    Method and device for generating a 3D reconstruction of a scene with a hybrid camera rig

    公开(公告)号:US12219118B1

    公开(公告)日:2025-02-04

    申请号:US17678815

    申请日:2022-02-23

    Applicant: Apple Inc.

    Abstract: In one implementation, a camera rig comprises: a first array of image sensors arranged in a planar configuration, wherein the first array of image sensors is provided to capture a first image stream from a first perspective of a physical environment; a second array of image sensors arranged in a non-planar configuration, wherein the second array of image sensors is provided to capture a second image stream from a second perspective of the physical environment different from the first perspective; a buffer provided to store the first and second image streams; and an image processing engine provided to generate a 3D reconstruction of the physical environment based on the first and second image streams.

    Gaze-driven recording of video
    14.
    发明授权

    公开(公告)号:US11825103B2

    公开(公告)日:2023-11-21

    申请号:US17825167

    申请日:2022-05-26

    Applicant: Apple Inc.

    CPC classification number: H04N19/23 G06F3/013 G11B27/02

    Abstract: Systems and methods for gaze-driven recording of video are described. Some implementations may include accessing gaze data captured using one or more gaze-tracking sensors; applying a temporal filter to the gaze data to obtain a smoothed gaze estimate; determining a region of interest based on the smoothed gaze estimate, wherein the region of interest identifies a subset of a field of view; accessing a frame of video; recording a portion of the frame associated with the region of interest as an enhanced frame of video, wherein the portion of the frame corresponds to a smaller field of view than the frame; and storing, transmitting, or displaying the enhanced frame of video.

    Viewport adaptive volumetric content streaming and/or rendering

    公开(公告)号:US11418769B1

    公开(公告)日:2022-08-16

    申请号:US17222872

    申请日:2021-04-05

    Applicant: Apple Inc.

    Abstract: A system comprises an encoder configured to compress and encode data for three-dimensional volumetric content. The encoder also is configured to segment the three-dimensional volumetric content based on viewing areas, wherein different ones of the viewing areas correspond to visible portions of the volumetric content. The system may provide metadata to a client device to support viewport adaptive rendering of the three-dimensional volumetric content or may adaptively stream portions of the three-dimensional volumetric content to a rending device based on viewing areas of the three-dimensional volumetric content that are to be rendered at the rendering device.

    Computer-generated reality recorder

    公开(公告)号:US12299980B2

    公开(公告)日:2025-05-13

    申请号:US18380622

    申请日:2023-10-16

    Applicant: Apple Inc.

    Abstract: Implementations of the subject technology provides analyzing a recording of content. The subject technology generates metadata information based at least in part on the analyzing. The subject technology identifies, based at least in part on at least one of a user preference or a detected event, a region of interest or an object of interest in the recording of content. Based at least in part on the identified region of interest or object of interest, the subject technology generates a modified version of the recording of content. Further, the subject technology stores the modified version of the recording of content for subsequent playback on an electronic device.

    Recording Content in a Head-mounted Device

    公开(公告)号:US20240406364A1

    公开(公告)日:2024-12-05

    申请号:US18644222

    申请日:2024-04-24

    Applicant: Apple Inc.

    Abstract: A head-mounted device is provided that includes a variety of subsystems for generating extended reality content, displaying the extended reality content, and recording the extended reality content. The device may include a graphics rendering pipeline configured to render virtual content, tracking sensors configured to obtain user tracking information, a virtual content compositor configured to composite virtual frames based on the virtual content and the user tracking information, cameras configured to capture a video feed, a media merging compositor configured to overlay the composited virtual frames and the video feed, and a recording pipeline configured to record parameters, metadata, raw content, and/or adjusted content in an extended reality recording file. The extended reality recording file may have multiple discrete portions that may each be individually edited. The extended reality recording file may be used to present a replay on the head-mounted device and/or may be exported to an external device.

    Resolution budgeting by area for immersive video rendering

    公开(公告)号:US11861788B1

    公开(公告)日:2024-01-02

    申请号:US17347404

    申请日:2021-06-14

    Applicant: Apple Inc.

    CPC classification number: G06T15/08 G06T7/50 G06T9/00 G06T2207/10016

    Abstract: One or more computing devices implement a mesh analysis for evaluating meshes to be rendered when rendering immersive content. The mesh analysis identifies objects in a three-dimensional scene and determines geometrical complexity values for the objects. Objects with similar geometrical complexities are grouped into areas and a mesh vertices budget is determined for the respective areas. Metadata indicating the area definitions and corresponding mesh vertices budgets are generated. The metadata may be uploaded to a server to simplify meshes in the scene prior to streaming to a client, or the metadata may be provided to a client for use in simplifying the meshes as part of rendering the scene.

Patent Agency Ranking