公开(公告)号：WO2023018369A2

公开(公告)日：2023-02-16

申请号：PCT/SG2022/050531

申请日：2022-07-26

Applicant: LEMON INC.

Inventor： TAY, Michael Leong Hou ,  MA, Wanchun ,  CHENG, Shuo ,  WANG, Chao ,  LUO, Linjie

IPC: G06V40/16 ,  G06V10/82 ,  G06N3/09 ,  G06T7/246 ,  G06F18/2193 ,  G06T13/40 ,  G06T13/80 ,  G06T2207/20084 ,  G06T2207/30201 ,  G06T7/251 ,  G06V10/242 ,  G06V40/171 ,  G06V40/176

Abstract: The present disclosure describes techniques for facial expression recognition. A first loss function may be determined based on a first set of feature vectors associated with a first set of images depicting facial expressions and a first set of labels indicative of the facial expressions. A second loss function may be determined based on a second set of feature vectors associated with a second set of images depicting asymmetric facial expressions and a second set of labels indicative of the asymmetric facial expressions. The first loss function and the second loss function may be used to determine a maximum loss function. The maximum loss function may be applied during training of a model. The trained model may be configured to predict at least one asymmetric facial expression in a subsequently received image.

公开(公告)号：WO2022240354A1

公开(公告)日：2022-11-17

申请号：PCT/SG2022/050221

申请日：2022-04-14

Applicant: LEMON INC.

Inventor： LUO, Linjie ,  SONG, Guoxian ,  LIU, Jing ,  MA, Wanchun

IPC: G06N3/04 ,  G06N3/08 ,  G06V10/77 ,  G06K9/62 ,  G06F18/214 ,  G06N3/045 ,  G06N3/047 ,  G06N3/088 ,  G06T11/00 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30201 ,  G06T3/0006 ,  G06T3/0012 ,  G06T5/00

Abstract: Systems and method directed to an inversion-consistent transfer learning framework for generating portrait stylization using only limited exemplars. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be provided to a generative adversarial network (GAN) generator to generate a stylized image. In examples, the variational autoencoder is trained using a plurality of images while keeping the weights of a pre-trained GAN generator fixed, where the pre-trained GAN generator acts as a decoder for the encoder. In other examples, a multi-path attribute aware generator is trained using a plurality of exemplar images and learning transfer using the pre-trained GAN generator.

公开(公告)号：WO2023080845A2

公开(公告)日：2023-05-11

申请号：PCT/SG2022/050801

申请日：2022-11-04

Applicant: LEMON INC.

Inventor： LIU, Jing ,  LAI, Chunpong ,  SONG, Guoxian ,  LUO, Linjie

Abstract: Systems and methods directed to controlling the similarity between stylized portraits and an original photo are described. In examples, an input image is received and encoded using a variational autoencoder to generate a latent vector. The latent vector may be blended with latent vectors that best represent a face in the original user portrait image. The resulting blended latent vector may be provided to a generative adversarial network (GAN) generator to generate a controlled stylized image. In examples, one or more layers of the stylized GAN generator may be swapped with one or more layers of the original GAN generator. Accordingly, a user can interactively determine how much stylization vs. personalization should be included in a resulting stylized portrait.

公开(公告)号：WO2023063879A2

公开(公告)日：2023-04-20

申请号：PCT/SG2022/050703

申请日：2022-09-29

Applicant: LEMON INC.

Inventor： LIU, Jing ,  LAI, Chunpong ,  SONG, Guoxian ,  LUO, Linjie ,  YUAN, Ye

Abstract: Systems and method directed to generating a stylized image are disclosed. In particular, the method includes, in a first data path, (a) applying first stylization to an input image and (b) applying enlargement to the stylized image from (a). The method also includes, in a second data path, (c) applying segmentation to the input image to identify a face region of the input image and generate a mask image, and (d) applying second stylization to an entirety of the input image and inpainting to the identified face region of the stylized image. Machine-assisted blending is performed based on (1) the stylized image after the enlargement from the first data path, (2) the inpainted image from the second data path, and (3) the mask image, in order to obtain a final stylized image.

公开(公告)号：WO2023009060A1

公开(公告)日：2023-02-02

申请号：PCT/SG2022/050345

申请日：2022-05-24

Applicant: LEMON INC.

Inventor： MA, Wanchun ,  CHENG, Shuo ,  WANG, Chao ,  TAY, Michael Leong Hou ,  LUO, Linjie

IPC: G06V10/82 ,  G06V40/16 ,  G06N3/0464

Abstract: The present disclosure describes techniques for face tracking. The techniques comprise receiving landmark data associated with a plurality of images indicative of at least one facial part. Representative images corresponding to the plurality of images may be generated based on the landmark data. Each representative image may depict a plurality of segments, and each segment may correspond to a region of the at least one facial part. The plurality of images and corresponding representative images may be input into a neural network to train the neural network to predict a feature associated with a subsequently received image comprising a face. An animation associated with a facial expression may be controlled based on output from the trained neural network.

公开(公告)号：EP4268136A1

公开(公告)日：2023-11-01

申请号：EP22807950.5

申请日：2022-04-14

Applicant: Lemon Inc.

Inventor： LUO, Linjie ,  SONG, Guoxian ,  LIU, Jing ,  MA, Wanchun

IPC: G06N3/04 ,  G06N3/08 ,  G06V10/77 ,  G06K9/62

公开(公告)号：EP4246435A1

公开(公告)日：2023-09-20

申请号：EP21907253.5

申请日：2021-11-24

Applicant: Lemon Inc.

Inventor： GAO, Yaxi ,  SUN, Chenyu ,  YANG, Xiao ,  CHEN, Zhili ,  LUO, Linjie ,  LIU, Jing ,  GUO, Hengkai ,  LI, Huaxia ,  KIM, Hwankyoo Shawn ,  YANG, Jianchao

IPC: G06T7/10 ,  G02B27/01 ,  G06F3/01 ,  G06T19/20

Abstract: Embodiments of the present disclosure provide a display method and apparatus based on augmented reality, a device, and a storage medium, the method includes receiving a first video; acquiring a video material by segmenting a target object from the first video; acquiring and displaying a real scene image, where the real scene image is acquired by an image collection apparatus; and displaying the video material at a target position of the real scene image in an augmented manner and playing the video material. Since the video material is acquired by receiving the first video and segmenting the target object from the first video, the video material may be set according to the needs of the user, so as to meet the purpose that the user customizes the loading and displaying of the video material, the customized video material is displayed on the real scene image in a manner of augmented reality, forming the video effect that align with the user's conception, enhancing the flexibility of the video creation, and improving the video expressiveness.