ENVIRONMENT-MODELING BASED CAMERA ADAPTATION FOR PASSTHROUGH EXTENDED REALITY SYSTEMS

    公开(公告)号:US20250111589A1

    公开(公告)日:2025-04-03

    申请号:US18826434

    申请日:2024-09-06

    Applicant: Apple Inc.

    Abstract: Various implementations provide passthrough video based on adjusting camera parameters based on environment modeling. An environment characteristic may be determined based on modeling the physical environment based on sensor data captured via one or more sensors. For example, this may involve determining environment light source optical characteristics, environment surfaces optical characteristics, a 3D mapping of the environment, user behavior, a prediction of optical characteristics of light coming in the camera, and the like. The method may involve, based on the environment characteristic, determining a camera parameter for an image captured via the image sensor. For example, the method may determine exposure, gain, tone mapping, color balance, noise reduction, sharpness enhancement. The method may determine the camera parameter based on user information, e.g., user preferences, user activity, etc. The method may involve providing passthrough video of the physical environment based on the determined camera parameter.

    Late-Stage Warp of Dynamic-Anchored Content

    公开(公告)号:US20240404230A1

    公开(公告)日:2024-12-05

    申请号:US18672992

    申请日:2024-05-23

    Applicant: Apple Inc.

    Abstract: In one implementation, a method of displaying an image is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of an object in a physical environment. The method includes obtaining, based on the image, a first predicted object pose of the object in the physical environment at a display time. The method includes rendering virtual content based on the first predicted object pose. The method includes obtaining a second predicted object pose of the object in the physical environment at the display time. The method includes warping the virtual content based on the second predicted object pose. The method includes displaying, on the display at the display time, the warped virtual content.

    Rendering Layers with Different Perception Quality

    公开(公告)号:US20240404165A1

    公开(公告)日:2024-12-05

    申请号:US18672922

    申请日:2024-05-23

    Applicant: Apple Inc.

    Abstract: In one implementation, a method of displaying image is performed by a device including one or more processors and non-transitory memory. The method includes obtaining gaze information. The method includes obtaining, based on the gaze information, a first resolution function and a second resolution function different than the first resolution function. The method includes rendering a first layer based on first virtual content and the first resolution function. The method includes rendering a second layer based on second virtual content and the second resolution function. The method includes compositing the first layer and the second layer into an image. The method includes displaying, on the display, the image.

    Stereoscopic Foveated Image Generation
    17.
    发明公开

    公开(公告)号:US20240267503A1

    公开(公告)日:2024-08-08

    申请号:US18437143

    申请日:2024-02-08

    Applicant: Apple Inc.

    CPC classification number: H04N13/296 G06T3/40 G06T15/00 H04N13/344

    Abstract: In one implementation, a method of generating an image is performed by a device including one or more processors and non-transitory memory. The method includes generating a first resolution function based on a formula with a set of variables having a first set of values. The method includes generating a first image based on first content and the first resolution function. The method includes detecting a resolution constraint. The method includes generating a second resolution function based on the formula with the set of variables having a second set of values, wherein the second resolution function has a summation value that satisfies the resolution constraint. The method includes generating a second image based on second content and the second resolution function.

    Methods and devices for improved inverse iterative warping

    公开(公告)号:US12014472B1

    公开(公告)日:2024-06-18

    申请号:US16983201

    申请日:2020-08-03

    Applicant: Apple Inc.

    CPC classification number: G06T3/18 G06T19/20 G06T2219/2021

    Abstract: In some implementations, a method includes: obtaining a reference image frame and forward flow information; for a respective pixel within a target image frame, obtaining a plurality of starting points within the reference image frame with different depths; generating a plurality of intermediate warp results based on the plurality of starting points and the forward flow information, wherein each of the plurality of intermediate warp results is associated with a candidate warp position and an associated depth; selecting a warp result for the respective pixel from among the plurality of intermediate warp results, wherein the warp result corresponds to the candidate warp position associated with a closest depth to a viewpoint associated with the reference image frame; and populating pixel information for the respective pixel within the target image frame based on pixel information for a reference pixel within the reference image frame that corresponds to the warp result.

Patent Agency Ranking