Method and system for determining spatial coordinates of a 3D reconstruction of at least part of a real object at absolute spatial scale

    公开(公告)号:US10445895B2

    公开(公告)日:2019-10-15

    申请号:US15525310

    申请日:2014-11-21

    Applicant: Apple Inc.

    Abstract: The invention, in one aspect, is related to a method of determining spatial coordinates of a 3D reconstruction of at least part of a first real object at absolute spatial scale comprising the steps of: receiving image information of a first image including at least part of the first real object captured with a first camera, of a third image including the at least part of the first real object captured with a third camera, of a second image including at least a first part of a second real object captured with a second camera, and of a fourth image including at least a second part of the second real object captured with a fourth camera, wherein the frustum of the first camera and the frustum of the second camera do not overlap and wherein the frustum of the third camera and the frustum of the fourth camera do not overlap, and providing a first spatial transformation between the first camera and the second camera and providing a second spatial transformation between the third camera and the fourth camera, providing a first scale information indicative of an absolute spatial scale of the at least first part of the second real object, and a second scale information indicative of an absolute spatial scale of the at least second part of the second real object, determining at least part of a pose (P2F) of the second camera and at least part of a pose (P4F) of the fourth camera according to the second image, the first scale information, the fourth image and the second scale information, wherein the at least part of the pose (P2F) of the second camera and the at least part of the pose (P4F) of the fourth camera are defined in a second common coordinate system, determining at least part of a pose (P1F) of the first camera according to the at least part of the pose (P2F) of the second camera and the first spatial transformation, and determining at least part of a pose (P3F) of the third camera according to the at least part of the pose (P4F) of the fourth camera and the second spatial transformation, and determining spatial coordinates of a 3D reconstruction of the at least part of the first real object at absolute spatial scale according to the first image, the third image and the at least part of the pose (P1F) of the first camera and the at least part of the pose (P3F) of the third camera.

    Method and system for generating at least one image of a real environment

    公开(公告)号:US20190230297A1

    公开(公告)日:2019-07-25

    申请号:US16371568

    申请日:2019-04-01

    Applicant: Apple Inc.

    Abstract: A method of generating at least one image of a real environment comprises providing at least one environment property related to at least part of the real environment, providing at least one virtual object property related to a virtual object, determining at least one imaging parameter according to the at least one provided virtual object property and the at least one provided environment property, and generating at least one image of the real environment representing information about light leaving the real environment according to the determined at least one imaging parameter, wherein the light leaving the real environment is measured by at least one camera.

    Method of Providing Image Feature Descriptors

    公开(公告)号:US20190156143A1

    公开(公告)日:2019-05-23

    申请号:US16259367

    申请日:2019-01-28

    Applicant: Apple Inc.

    CPC classification number: G06K9/623 G06K9/46 G06K9/4671 G06K9/6255 G06K9/6262

    Abstract: A method of providing a set of feature descriptors configured to be used in matching an object in an image of a camera is provided. The method includes: a) providing at least two images of a first object; b) extracting in at least two of the images at least one feature from the respective image, c) providing at least one descriptor for an extracted feature, and storing the descriptors; d) matching descriptors in the first set of descriptors; e) computing a score parameter based on the result of the matching process; f) selecting at least one descriptor based on its score parameter; g) adding the selected descriptor(s) to a second set of descriptors; and h) updating the score parameter of descriptors in the first set based on a selection process and to the result of the matching process.

    METHOD AND SYSTEM FOR DETERMINING AT LEAST ONE PROPERTY RELATED TO AT LEAST PART OF A REAL ENVIRONMENT

    公开(公告)号:US20180268237A1

    公开(公告)日:2018-09-20

    申请号:US15516548

    申请日:2014-10-01

    Applicant: Apple Inc.

    Abstract: A method for determining at least one property related to at least part of a real environment comprises receiving a first image of a first part of a real environment captured by a first camera, wherein the first camera is a thermal camera and the first image is a thermal image and the first part of the real environment is a first environment part, providing at least one description related to at least one class of real objects, wherein the at least one description includes at least one thermal property related to the at least one class of real objects, receiving a second image of the first environment part and of a second part of the real environment captured by a second camera, wherein the second part of the real environment is a second environment part, providing an image alignment between the first image and the second image, determining, for at least one second image region contained in the second image, at least one second probability according to the image alignment, pixel information of the first image, and the at least one description, wherein the at least one second probability relates to the at least one class of real objects, and wherein the at least one second image region comprises at least part of the first environment part, determining at least one image feature derived from pixel information of at least one third image region contained in the second image according to the at least one second probability, wherein the at least one third image region comprises at least part of the second environment part, and performing at least one computer vision method to determine at least one property related to at least part of the second environment part according to the determined at least one image feature.

    Detecting specular surfaces
    15.
    发明授权

    公开(公告)号:US12073533B1

    公开(公告)日:2024-08-27

    申请号:US17820442

    申请日:2022-08-17

    Applicant: Apple Inc.

    CPC classification number: G06T3/60 G06V10/60 G06T2207/30244

    Abstract: Identifying a specular surface, such as a mirror, in a captured scene includes extracting, from one or more images of the scene, a set of natural features and generating, from the image, a set of synthesized “mirrored” features. One or more correspondences may be determined between the set of natural features in the image and the set of synthesized mirrored features. A first set of features are identified based on the determined one or more correspondences as representing a specular surface (e.g., a mirror) located in the scene, and then a geometry and/or location of the specular surface within the scene may be determined. For example, in some embodiments, the feature from a determined pair of corresponding features in a scene that is determined to be farther away from the device that captured the image(s) of the scene may be determined to be the feature lying on the specular surface.

    Transparent insert identification
    16.
    发明授权

    公开(公告)号:US11948043B2

    公开(公告)日:2024-04-02

    申请号:US17824339

    申请日:2022-05-25

    Applicant: Apple Inc.

    Abstract: Various implementations disclosed herein include devices, systems, and methods that use a marking on a transparent surface (e.g., a prescription lens insert for an HMD) to identify information (e.g., prescription parameters) about the transparent surface. In some implementations, the markings do not interfere with eye tracking through the transparent surface or using the transparent surface to view virtual content or a physical environment. In some implementations, image data is obtained from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device. Then, a code is identified in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with a function of the electronic device involving the transparent surface. In some implementations, content is provided at the electronic device based on the identified code, wherein the content is viewable through the transparent surface.

    Method and device for detecting a touch between a first object and a second object

    公开(公告)号:US11797132B2

    公开(公告)日:2023-10-24

    申请号:US17133043

    申请日:2020-12-23

    Applicant: Apple Inc.

    Inventor: Daniel Kurz

    Abstract: The present disclosure is related to a method and device for detecting a touch between at least part of a first object and at least part of a second object, wherein the at least part of the first object has a different temperature than the at least part of the second object. The method includes providing at least one thermal image of a portion of the second object, determining in at least part of the at least one thermal image a pattern which is indicative of a particular value or range of temperature or a particular value or range of temperature change, and using the determined pattern for detecting a touch between the at least part of the first object and the at least part of the second object.

    METHOD AND DEVICE FOR EYE TRACKING USING EVENT CAMERA DATA

    公开(公告)号:US20230314798A1

    公开(公告)日:2023-10-05

    申请号:US17961963

    申请日:2022-10-07

    Applicant: Apple Inc.

    Abstract: In one implementation, a method includes emitting light with modulating intensity from a plurality of light sources towards an eye of a user. The method includes receiving light intensity data indicative of an intensity of the plurality of glints reflected by the eye of the user in the form of a plurality of glints. The method includes determining an eye tracking characteristic of the user based on the light intensity data. In one implementation, a method includes generating, using an event camera comprising a plurality of light sensors at a plurality of respective locations, a plurality of event messages, each of the plurality of event messages being generated in response to a particular light sensor detecting a change in intensity of light and indicating a particular location of the particular light sensor. The method includes determining an eye tracking characteristic of a user based on the plurality of event messages.

    Environment synthesis for lighting an object

    公开(公告)号:US11694392B2

    公开(公告)日:2023-07-04

    申请号:US16372882

    申请日:2019-04-02

    Applicant: Apple Inc.

    CPC classification number: G06T15/50 G06T7/40 G06T7/73 G06T19/006

    Abstract: Various implementations disclosed herein include devices, systems, and methods that render a reflective surface of a computer-generated reality (“CGR”) object based on synthesis in a CGR environment. In order to render a reflective surface of the CGR object, one exemplary implementation involves synthesizing an environment map of a CGR environment representing a portion of a physical scene based on observed characteristics of the physical scene. In an implementation, generation of a complete environment map includes identifying pixels of the environment map with no corresponding texture and generating synthesized texture based on textural information associated with one or more camera images of the physical scene. In an implementation, a CGR object is rendered in the CGR environment, wherein an appearance of a reflective surface of the CGR object is determined based on the complete environment map of the CGR environment.

Patent Agency Ranking