公开(公告)号：US11836838B2

公开(公告)日：2023-12-05

申请号：US17111083

申请日：2020-12-03

Applicant: Apple Inc.

Inventor： Thibaut Weise ,  Sofien Bouaziz ,  Hao Li ,  Mark Pauly

IPC: G06T13/40 ,  G06V40/16 ,  G06F18/2135 ,  G06F18/2415 ,  G06V10/764 ,  G06V10/77

CPC classification number: G06T13/40 ,  G06F18/2135 ,  G06F18/2415 ,  G06V10/764 ,  G06V10/7715 ,  G06V40/176

Abstract: A method of animating a digital character according to facial expressions of a user, comprising the steps of, (a) obtaining a 2D image and 3D depth map of the face of the user, (b) determining expression parameters for a user expression model so that a facial expression of the user-specific expression model represents the face of the user shown in the 2D image and 3D depth map (c) using the expression parameters and an animation prior to determine animation parameters usable to animate a digital character, wherein the animation prior is a sequence of animation parameters which represent predefined animations of a digital character (d) using the animation parameters to animate a digital character so that the digital character mimics the face of the user.

公开(公告)号：US11120600B2

公开(公告)日：2021-09-14

申请号：US16276535

申请日：2019-02-14

Applicant: Apple Inc.

Inventor： Justin D. Stoyles ,  Alexandre R. Moha ,  Nicolas V. Scapel ,  Guillaume P. Barlier ,  Aurelio Guzman ,  Bruno M. Sommer ,  Nina Damasky ,  Thibaut Weise ,  Thomas Goossens ,  Hoan Pham ,  Brian Amberg

IPC: G06T13/40 ,  G06K9/00 ,  H04N7/14 ,  G11B27/036 ,  G11B27/10 ,  G06T13/20 ,  G06T17/20

Abstract: Systems and methods for generating a video of an emoji that has been puppeted using inputs from image, depth, and audio. The inputs can capture facial expressions of a user, eye, eyebrow, mouth, and head movements. A pose, held by the user, can be detected that can be used to generate supplemental animation. The emoji can further be animated using physical properties associated with the emoji and captured movements. An emoji of a dog can have its ears move in response to an up-and-down movement, or a shaking of the head. The video can be sent in a message to one or more recipients. A sending device can render the puppeted video in accordance with hardware and software capabilities of a recipient's computer device.

公开(公告)号：US10430642B2

公开(公告)日：2019-10-01

申请号：US15934521

申请日：2018-03-23

Applicant: Apple Inc.

Inventor： Sofien Bouaziz ,  Brian Amberg ,  Thibaut Weise ,  Patrick Snape ,  Stefan Brugger ,  Alex Mansfield ,  Reinhard Knothe ,  Thomas Kiser

IPC: G06T15/00 ,  G06K9/00 ,  G06T17/20

Abstract: A three-dimensional model (e.g., motion capture model) of a user is generated from captured images or captured video of the user. A machine learning network may track poses and expressions of the user to generate and refine the three-dimensional model. Refinement of the three-dimensional model may provide more accurate tracking of the user's face. Refining of the three-dimensional model may include refining the determinations of poses and expressions at defined locations (e.g., eye corners and/or nose) in the three-dimensional model. The refining may occur in an iterative process. Tracking of the three-dimensional model over time (e.g., during video capture) may be used to generate an animated three-dimensional model (e.g., an animated puppet) of the user that simulates the user's poses and expressions.

公开(公告)号：US20180336714A1

公开(公告)日：2018-11-22

申请号：US15809875

申请日：2017-11-10

Applicant: Apple Inc.

Inventor： Justin D. Stoyles ,  Alexandre R. Moha ,  Nicolas V. Scapel ,  Guillaume P. Barlier ,  Aurelio Guzman ,  Bruno M. Sommer ,  Nina Damasky ,  Thibaut Weise ,  Thomas Goossens ,  Hoan Pham ,  Brian Amberg

IPC: G06T13/40 ,  G06K9/00 ,  H04N7/14

CPC classification number: G06T13/40 ,  G06K9/00228 ,  G06K9/00302 ,  G06T13/205 ,  G06T17/205 ,  G11B27/036 ,  G11B27/10 ,  H04N7/147 ,  H04N2007/145

Abstract: Systems and methods for generating a video of an emoji that has been puppeted using inputs from image, depth, and audio. The inputs can capture facial expressions of a user, eye, eyebrow, mouth, and head movements. A pose, held by the user, can be detected that can be used to generate supplemental animation. The emoji can further be animated using physical properties associated with the emoji and captured movements. An emoji of a dog can have its ears move in response to an up-and-down movement, or a shaking of the head. The video can be sent in a message to one or more recipients. A sending device can render the puppeted video in accordance with hardware and software capabilities of a recipient's computer device.

公开(公告)号：US20240388687A1

公开(公告)日：2024-11-21

申请号：US18662994

申请日：2024-05-13

Applicant: Apple Inc.

Inventor： Michael J. Rockwell ,  Geoffrey Stahl ,  Thibaut Weise ,  Peter Kaufmann ,  Branko Petljanski ,  Jason L. Slupeiks ,  Tom Sengelaub ,  Kathrin Berkner Cieslicki ,  Yanghai Tsin ,  Hesam Najafi ,  Arthur Y. Zhang ,  Amanda R. Rainer ,  Cheng Chen ,  Graham B. Myhre ,  Fletcher R. Rothkopf ,  Marinus Meursing ,  Phil M. Hobson ,  Jason C. Sauers ,  Jan K. Quijalvo ,  Jia Tao ,  Ivan S. Maric ,  Jeremy C. Franklin ,  Wey-Jiun Lin ,  William A. Sorrentino, III ,  Jonathan Ive ,  Alan C. Dye ,  Stephen O. Lemay ,  Seung Wook Kim ,  Ray L. Chang

IPC: H04N13/398 ,  G06F3/01 ,  G06F3/044 ,  H04N13/293 ,  H04N13/344 ,  H04N13/383

Abstract: A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

公开(公告)号：US20240386679A1

公开(公告)日：2024-11-21

申请号：US18662906

申请日：2024-05-13

Applicant: Apple Inc.

Inventor： Michael J. Rockwell ,  Geoffrey Stahl ,  Thibaut Weise ,  Peter Kaufmann ,  Branko Petljanski ,  Jason L. Slupeiks ,  Tom Sengelaub ,  Kathrin Berkner Cieslicki ,  Yanghai Tsin ,  Hesam Najafi ,  Arthur Y. Zhang ,  Julian Hoenig ,  Julian Jaede ,  Jason C. Sauers ,  James W. Vandyke ,  Yoonhoo Jo ,  Forrest C. Wang ,  Cheng Chen ,  Graham B. Myhre ,  Fletcher R. Rothkopf ,  Marinus Meursing ,  Phil M. Hobson ,  Jan K. Quijalvo ,  Jia Tao ,  Ivan S. Maric ,  Jeremy C. Franklin ,  Wey-Jiun Lin ,  Bertrand Nepveu ,  Muhammad F. Hossain ,  William A. Sorrentino, III ,  Jonathan Ive ,  Alan C. Dye ,  Stephen O. Lemay

IPC: G06T19/00 ,  G06F1/16 ,  G06F3/01 ,  G06F3/02 ,  G06F3/0362 ,  H04N23/90

Abstract: A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

公开(公告)号：US20240385687A1

公开(公告)日：2024-11-21

申请号：US18662739

申请日：2024-05-13

Applicant: Apple Inc.

Inventor： Thibaut Weise ,  Peter Kaufmann ,  Branko Petljanski ,  Jason L. Slupeiks ,  Tom Sengelaub ,  Kathrin Berkner Cieslicki ,  Yanghai Tsin ,  Hesam Najafi ,  Arthur Y. Zhang

IPC: G06F3/01 ,  G02B27/01 ,  G06V40/16 ,  G06V40/18 ,  G06V40/20

Abstract: A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

公开(公告)号：US11068698B2

公开(公告)日：2021-07-20

申请号：US16586758

申请日：2019-09-27

Applicant: Apple Inc.

Inventor： Sofien Bouaziz ,  Brian Amberg ,  Thibaut Weise ,  Patrick Snape ,  Stefan Brugger ,  Alex Mansfield ,  Reinhard Knothe ,  Thomas Kiser

IPC: G06T15/00 ,  G06K9/00 ,  G06T17/20

Abstract: A three-dimensional model (e.g., motion capture model) of a user is generated from captured images or captured video of the user. A machine learning network may track poses and expressions of the user to generate and refine the three-dimensional model. Refinement of the three-dimensional model may provide more accurate tracking of the user's face. Refining of the three-dimensional model may include refining the determinations of poses and expressions at defined locations (e.g., eye corners and/or nose) in the three-dimensional model. The refining may occur in an iterative process. Tracking of the three-dimensional model over time (e.g., during video capture) may be used to generate an animated three-dimensional model (e.g., an animated puppet) of the user that simulates the user's poses and expressions.

公开(公告)号：US20210174567A1

公开(公告)日：2021-06-10

申请号：US17111083

申请日：2020-12-03

Applicant: Apple Inc.

Inventor： Thibaut Weise ,  Sofien Bouaziz ,  Hao Li ,  Mark Pauly

IPC: G06T13/40 ,  G06K9/00 ,  G06K9/62

Abstract: A method of animating a digital character according to facial expressions of a user, comprising the steps of, (a) obtaining a 2D image and 3D depth map of the face of the user, (b) determining expression parameters for a user expression model so that a facial expression of the user-specific expression model represents the face of the user shown in the 2D image and 3D depth map (c) using the expression parameters and an animation prior to determine animation parameters usable to animate a digital character, wherein the animation prior is a sequence of animation parameters which represent predefined animations of a digital character (d) using the animation parameters to animate a digital character so that the digital character mimics the face of the user.

公开(公告)号：US10820795B1

公开(公告)日：2020-11-03

申请号：US16015683

申请日：2018-06-22

Applicant: Apple Inc.

Inventor： Thibaut Weise ,  Justin D. Stoyles ,  Michael Kuhn ,  Reinhard Klapfer ,  Stefan Misslinger

IPC: A61B3/11 ,  G02C13/00 ,  G02B27/01 ,  G06K9/20 ,  G02B7/02

Abstract: In one implementation, a method includes: determining an interpupillary distance (IPD) measurement for a user based on a function of depth data obtained by the depth sensor and image data obtained by the image sensor; and calibrating a head-mounted device (HMD) provided to deliver augmented reality/virtual reality (AR/VR) content by setting one or more presentation parameters of the HMD based on the IPD measurement in order to tailor one or more AR/VR displays of the HMD to a field-of-view of the user.