公开(公告)号：US20240193412A1

公开(公告)日：2024-06-13

申请号：US18063843

申请日：2022-12-09

Applicant: Lemon Inc.

Inventor： Song Bai ,  Zhongcong Xu ,  Jiashi Feng ,  Jun Hao Liew ,  Wenqing Zhang

IPC: G06N3/08

CPC classification number: G06N3/08 ,  G06T2207/20081

Abstract: Generating a multi-dimensional video using a multi-dimensional video generative model for, including, but not limited to, at least one of static portrait animation, video reconstruction, or motion editing. The method including providing data into the multi-dimensionally aware generator of the multi-dimensional video generative model, and generating the multi-dimensional video from the data by the multi-dimensionally aware generator. The generating of the multi-dimensional video includes inverting the data into a latent space of the multi-dimensionally aware generator, synthesizing content of the multi-dimensional video using an appearance component of the multi-dimensionally aware generator and corresponding camera pose and formulating an intermediate appearance code, developing a synthesis layer for encoding a motion component of the multi-dimensionally aware generator at a plurality of timesteps and formulating an intermediate motion code, introducing temporal dynamics into the intermediate appearance code and the intermediate motion code, and generating multi-dimensionally aware spatio-temporal representations of the data.

公开(公告)号：US20250157150A1

公开(公告)日：2025-05-15

申请号：US18941402

申请日：2024-11-08

Applicant: Lemon Inc.

Inventor： Zhongcong Xu ,  Jianfeng Zhang ,  Jun Hao Liew ,  Jiashi Feng

IPC: G06T17/20 ,  G06T3/4053 ,  G06V10/44 ,  G06V10/82

Abstract: Embodiments of the present disclosure disclose an image generation method, an apparatus, an electronic device, and a storage medium. The method includes: determining three-dimensional representations of preset areas in a target object according to a noise vector, wherein the three-dimensional representations are used to represent features of points in a space, and the preset areas have different size percentages in the target object; determining a three-dimensional mesh model in a target posture according to posture control parameters of the preset areas; sampling corresponding areas in the three-dimensional mesh model respectively according to camera poses for the preset areas, to obtain sampling points corresponding to the preset areas; determining target features corresponding to the sampling points according to the three-dimensional representations of the preset areas; and rendering the preset areas according to the target features, to generate target images, wherein the target images contain the target object in the target posture.

公开(公告)号：US20250014233A1

公开(公告)日：2025-01-09

申请号：US18347366

申请日：2023-07-05

Applicant: Lemon Inc.

Inventor： Jiachun Pan ,  Hanshu Yan ,  Jiashi Feng ,  Jun Hao Liew

IPC: G06T11/00 ,  G06T5/00

Abstract: Methods of customizing generation of objects using diffusion models are provided. One or more parameters (e.g., a conditioning signal, network weights, or an initial or starting noise) of the diffusion model can be optimized by a backpropagation process, which can be performed by solving an augmented adjoint ordinary differential equation (ODE) based on an adjoint sensitivity method. The customized diffusion model can generate stylized objects, generate objects with specific visual effect(s), and provide adversary examples to audit security of an object generation system.

公开(公告)号：US20240144544A1

公开(公告)日：2024-05-02

申请号：US18050349

申请日：2022-10-27

Applicant: Lemon Inc.

Inventor： Jun Hao Liew ,  Hanshu Yan ,  Daquan Zhou ,  Jiashi Feng

IPC: G06T11/00 ,  G06F40/40 ,  G06T5/00 ,  G06T5/20

CPC classification number: G06T11/00 ,  G06F40/40 ,  G06T5/002 ,  G06T5/20 ,  G06T2207/20084

Abstract: Generating an object using a diffusion model includes obtaining a first input and a second input, and synthesizing an output object from the first input and the second input. The synthesizing of the output object includes generating a layout of the output object from the first input, injecting the second input as a content conditioner to the layout of the output object, and de-noising the layout of the output object injected with the content conditioner to generate a content of the output object.