公开(公告)号：US20250095393A1

公开(公告)日：2025-03-20

申请号：US18470778

申请日：2023-09-20

Applicant: ADOBE INC.

Inventor： Ziyan Yang ,  Kushal Kafle ,  Zhe Lin ,  Scott Cohen ,  Zhihong Ding

IPC: G06V20/70 ,  G06F40/205 ,  G06V10/25 ,  G06V10/774

Abstract: A method, apparatus, and non-transitory computer readable medium for image processing are described. Embodiments of the present disclosure obtain an image and an input text including a subject from the image and a location of the subject in the image. An image encoder encodes the image to obtain an image embedding. A text encoder encodes the input text to obtain a text embedding. An image processing apparatus based on the present disclosure generates an output text based on the image embedding and the text embedding. In some examples, the output text includes a relation of the subject to an object from the image and a location of the object in the image.

公开(公告)号：US20250054116A1

公开(公告)日：2025-02-13

申请号：US18929330

申请日：2024-10-28

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Elya Shechtman ,  Connelly Barnes ,  Jianming Zhang ,  Ning Xu ,  Sohrab Amirghodsi

IPC: G06T5/77 ,  G06T3/4046 ,  G06V10/40

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.

公开(公告)号：US12136250B2

公开(公告)日：2024-11-05

申请号：US17332734

申请日：2021-05-27

Applicant: Adobe Inc.

Inventor： Khoi Pham ,  Kushal Kafle ,  Zhe Lin ,  Zhihong Ding ,  Scott Cohen ,  Quan Tran

IPC: G06V10/75 ,  G06F18/214 ,  G06F18/25 ,  G06N3/08

Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that extract multiple attributes from an object portrayed in a digital image utilizing a multi-attribute contrastive classification neural network. For example, the disclosed systems utilize a multi-attribute contrastive classification neural network that includes an embedding neural network, a localizer neural network, a multi-attention neural network, and a classifier neural network. In some cases, the disclosed systems train the multi-attribute contrastive classification neural network utilizing a multi-attribute, supervised-contrastive loss. In some embodiments, the disclosed systems generate negative attribute training labels for labeled digital images utilizing positive attribute labels that correspond to the labeled digital images.

公开(公告)号：US12045963B2

公开(公告)日：2024-07-23

申请号：US18058630

申请日：2022-11-23

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Zhihong Ding ,  Luis Figueroa ,  Kushal Kafle

IPC: G06T5/77 ,  G06F3/04842 ,  G06F3/04845 ,  G06T3/20 ,  G06V10/70 ,  G06V10/86

CPC classification number: G06T5/77 ,  G06F3/04842 ,  G06F3/04845 ,  G06T3/20 ,  G06V10/768 ,  G06V10/86 ,  G06T2200/24 ,  G06T2207/20084 ,  G06T2207/20104

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems detect, via a graphical user interface of a client device, a user selection of an object portrayed within a digital image. The disclosed systems determine, in response to detecting the user selection of the object, a relationship between the object and an additional object portrayed within the digital image. The disclosed systems receive one or more user interactions for modifying the object. The disclosed systems modify the digital image in response to the one or more user interactions by modifying the object and the additional object based on the relationship between the object and the additional object.

公开(公告)号：US20240169685A1

公开(公告)日：2024-05-23

申请号：US18058575

申请日：2022-11-23

Applicant: Adobe Inc.

Inventor： Luis Figueroa ,  Zhe Lin ,  Zhihong Ding ,  Scott Cohen

IPC: G06V10/20 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/60 ,  G06V10/82

CPC classification number: G06V10/255 ,  G06F3/04842 ,  G06F3/04845 ,  G06T11/60 ,  G06V10/82

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For instance, in one or more embodiments, the disclosed systems receive a digital image from a client device. The disclosed systems detect, utilizing a shadow detection neural network, an object portrayed in the digital image. The disclosed systems detect, utilizing the shadow detection neural network, a shadow portrayed in the digital image. The disclosed systems generate, utilizing the shadow detection neural network, an object-shadow pair prediction that associates the shadow with the object.

公开(公告)号：US20240169500A1

公开(公告)日：2024-05-23

申请号：US18058027

申请日：2022-11-22

Applicant: ADOBE INC.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jianming Zhang ,  Connelly Stuart Barnes ,  Elya Shechtman ,  Jingwan Lu ,  Qing Liu ,  Sohrab Amirghodsi ,  Yuqian Zhou ,  Scott Cohen

IPC: G06T5/00

CPC classification number: G06T5/005 ,  G06T5/003 ,  G06T2207/20081 ,  G06T2207/20104

Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image comprising a first region that includes content and a second region to be inpainted. Noise is then added to the image to obtain a noisy image, and a plurality of intermediate output images are generated based on the noisy image using a diffusion model trained using a perceptual loss. The intermediate output images predict a final output image based on a corresponding intermediate noise level of the diffusion model. The diffusion model then generates the final output image based on the intermediate output image. The final output image includes inpainted content in the second region that is consistent with the content in the first region.

公开(公告)号：US20240135514A1

公开(公告)日：2024-04-25

申请号：US18460365

申请日：2023-09-01

Applicant: Adobe Inc.

Inventor： Daniil Pakhomov ,  Qing Liu ,  Zhihong Ding ,  Scott Cohen ,  Zhe Lin ,  Jianming Zhang ,  Zhifei Zhang ,  Ohiremen Dibua ,  Mariette Souppe ,  Krishna Kumar Singh ,  Jonathan Brandt

IPC: G06T5/00 ,  G06F3/04845 ,  G06T7/11 ,  G06T7/194 ,  G06T7/70

CPC classification number: G06T5/005 ,  G06F3/04845 ,  G06T5/002 ,  G06T7/11 ,  G06T7/194 ,  G06T7/70 ,  G06T2200/24 ,  G06T2207/20021 ,  G06T2207/20084 ,  G06T2207/20092

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via multi-layered scene completion techniques facilitated by artificial intelligence. For instance, in some embodiments, the disclosed systems receive a digital image portraying a first object and a second object against a background, where the first object occludes a portion of the second object. Additionally, the disclosed systems pre-process the digital image to generate a first content fill for the portion of the second object occluded by the first object and a second content fill for a portion of the background occluded by the second object. After pre-processing, the disclosed systems detect one or more user interactions to move or delete the first object from the digital image. The disclosed systems further modify the digital image by moving or deleting the first object and exposing the first content fill for the portion of the second object.

公开(公告)号：US20240127412A1

公开(公告)日：2024-04-18

申请号：US17937708

申请日：2022-10-03

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Elya Shechtman ,  Jianming Zhang ,  Jingwan Lu ,  Ning Xu ,  Qing Liu ,  Scott Cohen ,  Sohrab Amirghodsi

IPC: G06T5/00 ,  G06T7/11

CPC classification number: G06T5/005 ,  G06T7/11 ,  G06T2207/20084 ,  G06T2207/20092

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：US11960843B2

公开(公告)日：2024-04-16

申请号：US16401548

申请日：2019-05-02

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Trung Huu Bui ,  Scott Cohen ,  Mingyang Ling ,  Chenyun Wu

IPC: G06N20/00 ,  G06F40/30 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82 ,  G06F18/21 ,  G06F40/205

CPC classification number: G06F40/30 ,  G06N20/00 ,  G06V10/25 ,  G06V10/764 ,  G06V10/82 ,  G06F18/217 ,  G06F40/205

Abstract: Techniques and systems are provided for training a machine learning model using different datasets to perform one or more tasks. The machine learning model can include a first sub-module configured to perform a first task and a second sub-module configured to perform a second task. The first sub-module can be selected for training using a first training dataset based on a format of the first training dataset. The first sub-module can then be trained using the first training dataset to perform the first task. The second sub-module can be selected for training using a second training dataset based on a format of the second training dataset. The second sub-module can then be trained using the second training dataset to perform the second task.

公开(公告)号：US11681919B2

公开(公告)日：2023-06-20

申请号：US17331161

申请日：2021-05-26

Applicant: Adobe Inc.

Inventor： Khoi Pham ,  Scott Cohen ,  Zhe Lin ,  Zhihong Ding ,  Walter Wei Tuh Chang

IPC: G06V10/00 ,  G06N3/08 ,  G06F18/2113 ,  G06F18/214 ,  G06F18/21 ,  G06V10/764 ,  G06V10/771 ,  G06V10/774 ,  G06V10/82

CPC classification number: G06N3/08 ,  G06F18/2113 ,  G06F18/2155 ,  G06F18/2163 ,  G06V10/764 ,  G06V10/765 ,  G06V10/771 ,  G06V10/7753 ,  G06V10/82

Abstract: The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image utilizing a large-scale object detector. For instance, in response to receiving a request to automatically select a query object with an unknown object class in a digital image, the object selection system can utilize a large-scale object detector to detect potential objects in the image, filter out one or more potential objects, and label the remaining potential objects in the image to detect the query object. In some implementations, the large-scale object detector utilizes a region proposal model, a concept mask model, and an auto tagging model to automatically detect objects in the digital image.