Invention Grant
- Patent Title: Remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice
-
Application No.: US17139792Application Date: 2020-12-31
-
Publication No.: US11620779B2Publication Date: 2023-04-04
- Inventor: Xiang Zhang , Xin Hou , Ken Lee , Yasmin Jahir
- Applicant: VanGogh Imaging, Inc.
- Applicant Address: US VA McLean
- Assignee: VanGogh Imaging, Inc.
- Current Assignee: VanGogh Imaging, Inc.
- Current Assignee Address: US VA McLean
- Agency: Cesari & McKenna, LLP
- Main IPC: G06T13/20
- IPC: G06T13/20 ; G06T7/246 ; G06T13/40 ; G06T7/90 ; G06T19/20 ; G06T13/80 ; G06V40/10 ; G06V40/16

Abstract:
Described herein are methods and systems for remote visualization of real-time three-dimensional (3D) facial animation with synchronized voice. A sensor captures frames of a face of a person, each frame comprising color images of the face, depth maps of the face, voice data associated with the person, and a timestamp. The sensor generates a 3D face model of the person using the depth maps. A computing device receives the frames of the face and the 3D face model. The computing device preprocesses the 3D face model. For each frame, the computing device: detects facial landmarks using the color images; matches the 3D face model to the depth maps using non-rigid registration; updates a texture on a front part of the 3D face model using the color images; synchronizes the 3D face model with a segment of the voice data using the timestamp; and transmits the synchronized 3D face model and voice data to a remote device.
Public/Granted literature
- US20210375020A1 REMOTE VISUALIZATION OF REAL-TIME THREE-DIMENSIONAL (3D) FACIAL ANIMATION WITH SYNCHRONIZED VOICE Public/Granted day:2021-12-02
Information query