-
1.发明公开公开(公告)号：US20230360320A1
公开(公告)日：2023-11-09
申请号：US18354619
申请日：2023-07-18
Applicant: Adobe Inc.
Inventor： Ruben Villegas , Yunseok Jang , Duygu Ceylan Aksit , Jimei Yang , Xin Sun
Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.
-
公开(公告)号：US11657546B2
公开(公告)日：2023-05-23
申请号：US17664800
申请日：2022-05-24
Applicant: Adobe Inc.
Inventor： Xin Sun , Ruben Villegas , Manuel Lagunas Arto , Jimei Yang , Jianming Zhang
CPC classification number: G06T11/001 , G06T7/11 , G06T7/194 , G06T7/90 , G06T2207/20084 , G06T2207/30196
Abstract: Introduced here are techniques for relighting an image by automatically segmenting a human object in an image. The segmented image is input to an encoder that transforms it into a feature space. The feature space is concatenated with coefficients of a target illumination for the image and input to an albedo decoder and a light transport detector to predict an albedo map and a light transport matrix, respectively. In addition, the output of the encoder is concatenated with outputs of residual parts of each decoder and fed to a light coefficients block, which predicts coefficients of the illumination for the image. The light transport matrix and predicted illumination coefficients are multiplied to obtain a shading map that can sharpen details of the image. Scaling the resulting image by the albedo map to produce the relight image. The relight image can be refined to denoise the relight image.
-
3.发明公开公开(公告)号：US20240346737A1
公开(公告)日：2024-10-17
申请号：US18756135
申请日：2024-06-27
Applicant: Adobe Inc.
Inventor： Jun Saito , Nitin Saini , Ruben Villegas
CPC classification number: G06T13/40 , G06T9/001 , G06T13/205 , G06T17/00
Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing unsupervised learning of discrete human motions to generate digital human motion sequences. The disclosed system utilizes an encoder of a discretized motion model to extract a sequence of latent feature representations from a human motion sequence in an unlabeled digital scene. The disclosed system also determines sampling probabilities from the sequence of latent feature representations in connection with a codebook of discretized feature representations associated with human motions. The disclosed system converts the sequence of latent feature representations into a sequence of discretized feature representations by sampling from the codebook based on the sampling probabilities. Additionally, the disclosed system utilizes a decoder to reconstruct a human motion sequence from the sequence of discretized feature representations. The disclosed system also utilizes a reconstruction loss and a distribution loss to learn parameters of the discretized motion model.
-
公开(公告)号：US11704865B2
公开(公告)日：2023-07-18
申请号：US17383294
申请日：2021-07-22
Applicant: Adobe Inc.
Inventor： Ruben Villegas , Yunseok Jang , Duygu Ceylan Aksit , Jimei Yang , Xin Sun
Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.
-
公开(公告)号：US20230037591A1
公开(公告)日：2023-02-09
申请号：US17383294
申请日：2021-07-22
Applicant: Adobe Inc.
Inventor： Ruben Villegas , Yunseok Jang , Duygu Ceylan Aksit , Jimei Yang , Xin Sun
Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.
-
Patent Agency Ranking
公开(公告)号：US11380023B2
公开(公告)日：2022-07-05
申请号：US16823092
申请日：2020-03-18
Applicant: Adobe Inc.
Inventor： Xin Sun , Ruben Villegas , Manuel Lagunas Arto , Jimei Yang , Jianming Zhang
Abstract: Introduced here are techniques for relighting an image by automatically segmenting a human object in an image. The segmented image is input to an encoder that transforms it into a feature space. The feature space is concatenated with coefficients of a target illumination for the image and input to an albedo decoder and a light transport detector to predict an albedo map and a light transport matrix, respectively. In addition, the output of the encoder is concatenated with outputs of residual parts of each decoder and fed to a light coefficients block, which predicts coefficients of the illumination for the image. The light transport matrix and predicted illumination coefficients are multiplied to obtain a shading map that can sharpen details of the image. Scaling the resulting image by the albedo map to produce the relight image. The relight image can be refined to denoise the relight image.
-
公开(公告)号：US10546408B2
公开(公告)日：2020-01-28
申请号：US15926787
申请日：2018-03-20
Applicant: Adobe Inc.
Inventor： Jimei Yang , Duygu Ceylan , Ruben Villegas
Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that use a motion synthesis neural network with a forward kinematics layer to generate a motion sequence for a target skeleton based on an initial motion sequence for an initial skeleton. In certain embodiments, the methods, non-transitory computer readable media, and systems use a motion synthesis neural network comprising an encoder recurrent neural network, a decoder recurrent neural network, and a forward kinematics layer to retarget motion sequences. To train the motion synthesis neural network to retarget such motion sequences, in some implementations, the disclosed methods, non-transitory computer readable media, and systems modify parameters of the motion synthesis neural network based on one or both of an adversarial loss and a cycle consistency loss.
-
公开(公告)号：US20190295305A1
公开(公告)日：2019-09-26
申请号：US15926787
申请日：2018-03-20
Applicant: Adobe Inc.
Inventor： Jimei Yang , Duygu Ceylan , Ruben Villegas
Abstract: This disclosure relates to methods, non-transitory computer readable media, and systems that use a motion synthesis neural network with a forward kinematics layer to generate a motion sequence for a target skeleton based on an initial motion sequence for an initial skeleton. In certain embodiments, the methods, non-transitory computer readable media, and systems use a motion synthesis neural network comprising an encoder recurrent neural network, a decoder recurrent neural network, and a forward kinematics layer to retarget motion sequences. To train the motion synthesis neural network to retarget such motion sequences, in some implementations, the disclosed methods, non-transitory computer readable media, and systems modify parameters of the motion synthesis neural network based on one or both of an adversarial loss and a cycle consistency loss.
-
公开(公告)号：US12067661B2
公开(公告)日：2024-08-20
申请号：US17651330
申请日：2022-02-16
Applicant: Adobe Inc.
Inventor： Jun Saito , Nitin Saini , Ruben Villegas
CPC classification number: G06T13/40 , G06T9/001 , G06T13/205 , G06T17/00
Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing unsupervised learning of discrete human motions to generate digital human motion sequences. The disclosed system utilizes an encoder of a discretized motion model to extract a sequence of latent feature representations from a human motion sequence in an unlabeled digital scene. The disclosed system also determines sampling probabilities from the sequence of latent feature representations in connection with a codebook of discretized feature representations associated with human motions. The disclosed system converts the sequence of latent feature representations into a sequence of discretized feature representations by sampling from the codebook based on the sampling probabilities. Additionally, the disclosed system utilizes a decoder to reconstruct a human motion sequence from the sequence of discretized feature representations. The disclosed system also utilizes a reconstruction loss and a distribution loss to learn parameters of the discretized motion model.
-
公开(公告)号：US12033261B2
公开(公告)日：2024-07-09
申请号：US17385559
申请日：2021-07-26
Applicant: Adobe Inc.
Inventor： Ruben Villegas , Jun Saito , Jimei Yang , Duygu Ceylan Aksit , Aaron Hertzmann
Abstract: One example method involves a processing device that performs operations that include receiving a request to retarget a source motion into a target object. Operations further include providing the target object to a contact-aware motion retargeting neural network trained to retarget the source motion into the target object. The contact-aware motion retargeting neural network is trained by accessing training data that includes a source object performing the source motion. The contact-aware motion retargeting neural network generates retargeted motion for the target object, based on a self-contact having a pair of input vertices. The retargeted motion is subject to motion constraints that: (i) preserve a relative location of the self-contact and (ii) prevent self-penetration of the target object.
-
-
-
-
-
-
-
-
-
Search Results
14
records
(took 0.014 seconds)
