-
公开(公告)号:US20250054199A1
公开(公告)日:2025-02-13
申请号:US18923108
申请日:2024-10-22
Applicant: Snap Inc.
Inventor: Jian Ren , Menglei Chai , Sergey Tulyakov , Qing Jin
Abstract: System and methods for compressing image-to-image models. Generative Adversarial Networks (GANs) have achieved success in generating high-fidelity images. An image compression system and method adds a novel variant to class-dependent parameters (CLADE), referred to as CLADE-Avg, which recovers the image quality without introducing extra computational cost. An extra layer of average smoothing is performed between the parameter and normalization layers. Compared to CLADE, this image compression system and method smooths abrupt boundaries, and introduces more possible values for the scaling and shift. In addition, the kernel size for the average smoothing can be selected as a hyperparameter, such as a 3×3 kernel size. This method does not introduce extra multiplications but only addition, and thus does not introduce much computational overhead, as the division can be absorbed into the parameters after training.
-
公开(公告)号:US20240282066A1
公开(公告)日:2024-08-22
申请号:US18653609
申请日:2024-05-02
Applicant: Snap Inc.
Inventor: Artem Bondich , Menglei Chai , Olekssandr Pyshchenko , Jian Ren , Sergey Tulyakov
CPC classification number: G06T19/006 , G06F18/214 , G06N3/045 , G06N3/08 , G06T7/251 , G06T17/00 , G06T19/20 , G06T2207/20081 , G06T2207/20084 , G06T2215/16 , G06T2219/024 , G06T2219/2024
Abstract: A messaging system performs neural network hair rendering for images provided by users of the messaging system. A method of neural network hair rendering includes processing a three-dimensional (3D) model of fake hair and a first real hair image depicting a first person to generate a fake hair structure, and encoding, using a fake hair encoder neural subnetwork, the fake hair structure to generate a coded fake hair structure. The method further includes processing, using a cross-domain structure embedding neural subnetwork, the coded fake hair structure to generate a fake and real hair structure, and encoding, using an appearance encoder neural subnetwork, a second real hair image depicting a second person having a second head to generate an appearance map. The method further includes processing, using a real appearance renderer neural subnetwork, the appearance map and the fake and real hair structure to generate a synthesized real image.
-
公开(公告)号:US20240221259A1
公开(公告)日:2024-07-04
申请号:US18149007
申请日:2022-12-30
Applicant: Snap Inc.
Inventor: Aleksandr Belskikh , Menglei Chai , Antoine Chassang , Anna Kovalenko , Pavel Savchenkov
CPC classification number: G06T13/40 , G06N3/094 , G06T19/006
Abstract: The subject technology generates a first image of a face using a GAN model. The subject technology applies 3D virtual hair on the first image to generate a second image with 3D virtual hair. The subject technology projects the second image with 3D virtual hair into a GAN latent space to generate a third image with realistic virtual hair. The subject technology performs a blend of the realistic virtual hair with the first image of the face to generate a new image with new realistic hair that corresponds to the 3D virtual hair. The subject technology trains a neural network that receives the second image with the 3D virtual hair and provides an output image with realistic virtual hair. The subject technology generates using the trained neural network, a particular output image with realistic hair based on a particular input image with 3D virtual hair.
-
公开(公告)号:US20240029346A1
公开(公告)日:2024-01-25
申请号:US17814063
申请日:2022-07-21
Applicant: Snap Inc.
Inventor: Zeng Huang , Menglei Chai , Sergey Tulyakov , Kyle Olszewski , Hsin-Ying Lee
CPC classification number: G06T17/00 , G06T15/04 , G06T2207/10028
Abstract: A system to enable 3D hair reconstruction and rendering from a single reference image which performs a multi-stage process that utilizes both a 3D implicit representation and a 2D parametric embedding space.
-
公开(公告)号:US20230394681A1
公开(公告)日:2023-12-07
申请号:US18235563
申请日:2023-08-18
Applicant: Snap Inc.
Inventor: Chen Cao , Menglei Chai , Linjie Luo , Oliver Woodford
IPC: G06T7/246 , G06T13/40 , G06T7/73 , G06V40/16 , G06V10/774
CPC classification number: G06T7/251 , G06T13/40 , G06T7/73 , G06V40/161 , G06V10/774 , G06V40/165 , G06V40/176 , G06V10/62
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.
-
公开(公告)号:US11798213B2
公开(公告)日:2023-10-24
申请号:US17364218
申请日:2021-06-30
Applicant: Snap Inc.
Inventor: Menglei Chai , Jian Ren , Aliaksandr Siarohin , Sergey Tulyakov , Oliver Woodford
CPC classification number: G06T13/00 , G06N3/045 , G06N3/08 , G06T7/20 , G06T11/001 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods herein describe novel motion representations for animating articulated objects consisting of distinct parts. The described systems and method access source image data, identify driving image data to modify image feature data in the source image sequence data, generate, using an image transformation neural network, modified source image data comprising a plurality of modified source images depicting modified versions of the image feature data, the image transformation neural network being trained to identify, for each image in the source image data, a driving image from the driving image data, the identified driving image being implemented by the image transformation neural network to modify a corresponding source image in the source image data using motion estimation differences between the identified driving image and the corresponding source image, and stores the modified source image data.
-
公开(公告)号:US11769259B2
公开(公告)日:2023-09-26
申请号:US17248908
申请日:2021-02-12
Applicant: Snap Inc.
Inventor: Chen Cao , Menglei Chai , Linjie Luo , Oliver Woodford
CPC classification number: G06T7/251 , G06T7/73 , G06T13/40 , G06V10/774 , G06V40/161 , G06V40/165 , G06V40/176 , G06V10/62
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex that is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.
-
公开(公告)号:US20230215085A1
公开(公告)日:2023-07-06
申请号:US18090091
申请日:2022-12-28
Applicant: Snap Inc.
Inventor: Kyle Olszewski , Sergey Tulyakov , Zhengfei Kuang , Menglei Chai
CPC classification number: G06T15/50 , G06T7/60 , G06T7/80 , G06T15/06 , G06T7/55 , G06T7/194 , G06T2210/12 , G06T2207/10028 , G06T2207/20084 , G06T2207/20081
Abstract: Three-dimensional object representation and re-rendering systems and methods for producing a 3D representation of an object from 2D images including the object that enables object-centric rendering. A modular approach is used that optimizes a Neural Radiance Field (NeRF) model to estimate object geometry and refine camera parameters and, then, infer surface material properties and per-image lighting conditions that fit the 2D images.
-
公开(公告)号:US10949648B1
公开(公告)日:2021-03-16
申请号:US16170997
申请日:2018-10-25
Applicant: Snap Inc.
Inventor: Chen Cao , Menglei Chai , Linjie Luo , Oliver Woodford
Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for accessing a set of images depicting at least a portion of a face. A set of facial regions of the face is identified, each facial region of the set of facial regions intersecting another facial region with at least one common vertex which is a member of a set of facial vertices. For each facial region of the set of facial regions, a weight formed from a set of region coefficients is generated. Based on the set of facial regions and the weight of each facial region of the set of facial regions, the face is tracked across the set of images.
-
公开(公告)号:US12056792B2
公开(公告)日:2024-08-06
申请号:US17557834
申请日:2021-12-21
Applicant: Snap Inc.
Inventor: Jian Ren , Menglei Chai , Oliver Woodford , Kyle Olszewski , Sergey Tulyakov
CPC classification number: G06T11/00 , G06N3/045 , G06T3/60 , G06T7/194 , G06V40/10 , G06T2207/20084 , G06T2207/30196
Abstract: Systems and methods herein describe a motion retargeting system. The motion retargeting system accesses a plurality of two-dimensional images comprising a person performing a plurality of body poses, extracts a plurality of implicit volumetric representations from the plurality of body poses, generates a three-dimensional warping field, the three-dimensional warping field configured to warp the plurality of implicit volumetric representations from a canonical pose to a target pose, and based on the three-dimensional warping field, generates a two-dimensional image of an artificial person performing the target pose.
-
-
-
-
-
-
-
-
-