Invention Grant
- Patent Title: Systems and methods for generating synthetic videos based on audio contents
-
Application No.: US17722258Application Date: 2022-04-15
-
Publication No.: US11968433B2Publication Date: 2024-04-23
- Inventor: Jing Zhao
- Applicant: Realsee (Beijing) Technology Co., Ltd.
- Applicant Address: CN Beijing
- Assignee: REALSEE (BEIJING) TECHNOLOGY CO., LTD.
- Current Assignee: REALSEE (BEIJING) TECHNOLOGY CO., LTD.
- Current Assignee Address: CN Beijing
- Agency: BAYES PLLC
- Priority: CN 2110437420.6 2021.04.22
- Main IPC: H04N21/8547
- IPC: H04N21/8547 ; G06N3/045 ; G06V20/40 ; G06V40/16 ; G11B27/031 ; G11B27/10 ; H04N21/845

Abstract:
Systems and methods for generating a synthetic video based on an audio are provided. An exemplary system may include a memory storing computer-readable instructions and at least one processor. The processor may execute the computer-readable instructions to perform operations. The operations may include receiving a reference video including a motion picture of a human face and receiving the audio including a speech. The operations may also include generating a synthetic motion picture of the human face based on the reference video and the audio. The synthetic motion picture of the human face may include a motion of a mouth of the human face presenting the speech. The motion of the mouth may match a content of the speech. The operations may further include generating the synthetic video based on the synthetic motion picture of the human face.
Public/Granted literature
- US20220345796A1 SYSTEMS AND METHODS FOR GENERATING SYNTHETIC VIDEOS BASED ON AUDIO CONTENTS Public/Granted day:2022-10-27
Information query