公开(公告)号：US12236610B2

公开(公告)日：2025-02-25

申请号：US17500736

申请日：2021-10-13

Applicant: Adobe Inc.

Inventor： Brian Price ,  Yutong Dai ,  He Zhang

IPC: G06T7/194 ,  G06T5/50 ,  G06T5/94 ,  G06T7/11

Abstract: This disclosure describes one or more implementations of an alpha matting system that utilizes a deep learning model to generate alpha mattes for digital images utilizing an alpha-range classifier function. More specifically, in various implementations, the alpha matting system builds and utilizes an object mask neural network having a decoder that includes an alpha-range classifier to determine classification probabilities for pixels of a digital image with respect to multiple alpha-range classifications. In addition, the alpha matting system can utilize a refinement model to generate the alpha matte from the pixel classification probabilities with respect to the multiple alpha-range classifications.

公开(公告)号：US20240404138A1

公开(公告)日：2024-12-05

申请号：US18205322

申请日：2023-06-02

Applicant: Adobe Inc.

Inventor： He Zhang ,  Shi Yan ,  Jianming Zhang

IPC: G06T11/40 ,  G06T7/11 ,  G06T7/90

Abstract: In accordance with the described techniques, an image delighting system receives an input image depicting a human subject that includes lighting effects. The image delighting system further generates a segmentation mask and a skin tone mask. The segmentation mask includes multiple segments each representing a different portion of the human subject, and the skin tone mask identifies one or more color values for a skin region of the human subject. Using a machine learning lighting removal network, the image delighting system generates an unlit image by removing the lighting effects from the input image based on the segmentation mask and the skin tone mask.

公开(公告)号：US12148074B2

公开(公告)日：2024-11-19

申请号：US17503671

申请日：2021-10-18

Applicant: Adobe Inc.

Inventor： He Zhang ,  Jeya Maria Jose Valanarasu ,  Jianming Zhang ,  Jose Ignacio Echevarria Vallespi ,  Kalyan Sunkavalli ,  Yilin Wang ,  Yinglan Ma ,  Zhe Lin ,  Zijun Wei

IPC: G06T11/60 ,  G06F3/04842 ,  G06F3/04845 ,  G06N3/08 ,  G06V10/40 ,  G06V10/75

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating harmonized digital images utilizing an object-to-object harmonization neural network. For example, the disclosed systems implement, and learn parameters for, an object-to-object harmonization neural network to combine a style code from a reference object with features extracted from a target object. Indeed, the disclosed systems extract a style code from a reference object utilizing a style encoder neural network. In addition, the disclosed systems generate a harmonized target object by applying the style code of the reference object to a target object utilizing an object-to-object harmonization neural network.

公开(公告)号：US20240362825A1

公开(公告)日：2024-10-31

申请号：US18762395

申请日：2024-07-02

Applicant: Adobe Inc.

Inventor： Brian Price ,  Yutong Dai ,  He Zhang

IPC: G06T9/00 ,  G06N3/04 ,  G06T3/4046 ,  G06T7/11 ,  G06T7/194

CPC classification number: G06T9/002 ,  G06N3/04 ,  G06T3/4046 ,  G06T7/11 ,  G06T7/194

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a transformer-based encoder-decoder neural network architecture for generating alpha mattes for digital images. Specifically, the disclosed system utilizes a transformer encoder to generate patch-based encodings from a digital image and a trimap segmentation by generating patch encodings for image patches and comparing the patch encodings to areas of the digital image. Additionally, the disclosed system generates modified patch-based encodings utilizing a plurality of neural network layers. The disclosed system also generates an alpha matte for the digital image from the patch-based encodings utilizing a decoder that includes a plurality of upsampling layers connected to a plurality of neural network layers via skip connections. In additional embodiments, the disclosed system generates the alpha matte based on additional encodings generated by a plurality of convolutional neural network layers connected to a subset of the upsampling layers via skip connections.

公开(公告)号：US20230122623A1

公开(公告)日：2023-04-20

申请号：US17503671

申请日：2021-10-18

Applicant: Adobe Inc.

Inventor： He Zhang ,  Jeya Maria Jose Valanarasu ,  Jianming Zhang ,  Jose Ignacio Echevarria Vallespi ,  Kalyan Sunkavalli ,  Yilin Wang ,  Yinglan Ma ,  Zhe Lin ,  Zijun Wei

IPC: G06T11/60 ,  G06F3/0484 ,  G06K9/46 ,  G06K9/62 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and flexibly generating harmonized digital images utilizing an object-to-object harmonization neural network. For example, the disclosed systems implement, and learn parameters for, an object-to-object harmonization neural network to combine a style code from a reference object with features extracted from a target object. Indeed, the disclosed systems extract a style code from a reference object utilizing a style encoder neural network. In addition, the disclosed systems generate a harmonized target object by applying the style code of the reference object to a target object utilizing an object-to-object harmonization neural network.

公开(公告)号：US20220012885A1

公开(公告)日：2022-01-13

申请号：US17483280

申请日：2021-09-23

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Jianming Zhang ,  He Zhang ,  Federico Perazzi

IPC: G06T7/10 ,  G06N3/04 ,  G06N3/08 ,  G06T11/60

Abstract: The present disclosure relates to utilizing a neural network having a two-stream encoder architecture to accurately generate composite digital images that realistically portray a foreground object from one digital image against a scene from another digital image. For example, the disclosed systems can utilize a foreground encoder of the neural network to identify features from a foreground image and further utilize a background encoder to identify features from a background image. The disclosed systems can then utilize a decoder to fuse the features together and generate a composite digital image. The disclosed systems can train the neural network utilizing an easy-to-hard data augmentation scheme implemented via self-teaching. The disclosed systems can further incorporate the neural network within an end-to-end framework for automation of the image composition process.

公开(公告)号：US20210027470A1

公开(公告)日：2021-01-28

申请号：US16523465

申请日：2019-07-26

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Jianming Zhang ,  He Zhang ,  Federico Perazzi

IPC: G06T7/10 ,  G06N3/04 ,  G06N3/08 ,  G06T11/60

Abstract: The present disclosure relates to utilizing a neural network having a two-stream encoder architecture to accurately generate composite digital images that realistically portray a foreground object from one digital image against a scene from another digital image. For example, the disclosed systems can utilize a foreground encoder of the neural network to identify features from a foreground image and further utilize a background encoder to identify features from a background image. The disclosed systems can then utilize a decoder to fuse the features together and generate a composite digital image. The disclosed systems can train the neural network utilizing an easy-to-hard data augmentation scheme implemented via self-teaching. The disclosed systems can further incorporate the neural network within an end-to-end framework for automation of the image composition process.

公开(公告)号：US20240404188A1

公开(公告)日：2024-12-05

申请号：US18205279

申请日：2023-06-02

Applicant: Adobe Inc.

Inventor： He Zhang ,  Zijun Wei ,  Zhixin Shu ,  Yiqun Mei ,  Yilin Wang ,  Xuaner Zhang ,  Shi Yan ,  Jianming Zhang

IPC: G06T15/50 ,  G06T7/11 ,  G06T15/04 ,  G06T15/60 ,  G06T15/80

Abstract: In accordance with the described techniques, a portrait relighting system receives user input defining one or more markings drawn on a portrait image. Using one or more machine learning models, the portrait relighting system generates an albedo representation of the portrait image by removing lighting effects from the portrait image. Further, the portrait relighting system generates a shading map of the portrait image using the one or more machine learning models by designating the one or more markings as a lighting condition, and applying the lighting condition to a geometric representation of the portrait image. The one or more machine learning models are further employed to generate a relit portrait image based on the albedo representation and the shading map.

公开(公告)号：US20240273813A1

公开(公告)日：2024-08-15

申请号：US18168995

申请日：2023-02-14

Applicant: Adobe Inc.

Inventor： Jianming Zhang ,  Yichen Sheng ,  Julien Philip ,  Yannick Hold-Geoffroy ,  Xin Sun ,  He Zhang

IPC: G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/74 ,  G06V10/82

CPC classification number: G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/761 ,  G06V10/82

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

公开(公告)号：US12051225B2

公开(公告)日：2024-07-30

申请号：US17513559

申请日：2021-10-28

Applicant: Adobe Inc.

Inventor： Brian Price ,  Yutong Dai ,  He Zhang

IPC: G06N3/04 ,  G06T3/4046 ,  G06T7/11 ,  G06T7/194 ,  G06T9/00

CPC classification number: G06T9/002 ,  G06N3/04 ,  G06T3/4046 ,  G06T7/11 ,  G06T7/194

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing a transformer-based encoder-decoder neural network architecture for generating alpha mattes for digital images. Specifically, the disclosed system utilizes a transformer encoder to generate patch-based encodings from a digital image and a trimap segmentation by generating patch encodings for image patches and comparing the patch encodings to areas of the digital image. Additionally, the disclosed system generates modified patch-based encodings utilizing a plurality of neural network layers. The disclosed system also generates an alpha matte for the digital image from the patch-based encodings utilizing a decoder that includes a plurality of upsampling layers connected to a plurality of neural network layers via skip connections. In additional embodiments, the disclosed system generates the alpha matte based on additional encodings generated by a plurality of convolutional neural network layers connected to a subset of the upsampling layers via skip connections.