公开(公告)号：US12204610B2

公开(公告)日：2025-01-21

申请号：US17650967

申请日：2022-02-14

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Haitian Zheng ,  Jingwan Lu ,  Scott Cohen ,  Jianming Zhang ,  Ning Xu ,  Elya Shechtman ,  Connelly Barnes ,  Sohrab Amirghodsi

IPC: G06K9/00 ,  G06F18/214 ,  G06N3/08 ,  G06T5/77 ,  G06T7/11

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for training a generative inpainting neural network to accurately generate inpainted digital images via object-aware training and/or masked regularization. For example, the disclosed systems utilize an object-aware training technique to learn parameters for a generative inpainting neural network based on masking individual object instances depicted within sample digital images of a training dataset. In some embodiments, the disclosed systems also (or alternatively) utilize a masked regularization technique as part of training to prevent overfitting by penalizing a discriminator neural network utilizing a regularization term that is based on an object mask. In certain cases, the disclosed systems further generate an inpainted digital image utilizing a trained generative inpainting model with parameters learned via the object-aware training and/or the masked regularization.

公开(公告)号：US20240404188A1

公开(公告)日：2024-12-05

申请号：US18205279

申请日：2023-06-02

Applicant: Adobe Inc.

Inventor： He Zhang ,  Zijun Wei ,  Zhixin Shu ,  Yiqun Mei ,  Yilin Wang ,  Xuaner Zhang ,  Shi Yan ,  Jianming Zhang

IPC: G06T15/50 ,  G06T7/11 ,  G06T15/04 ,  G06T15/60 ,  G06T15/80

Abstract: In accordance with the described techniques, a portrait relighting system receives user input defining one or more markings drawn on a portrait image. Using one or more machine learning models, the portrait relighting system generates an albedo representation of the portrait image by removing lighting effects from the portrait image. Further, the portrait relighting system generates a shading map of the portrait image using the one or more machine learning models by designating the one or more markings as a lighting condition, and applying the lighting condition to a geometric representation of the portrait image. The one or more machine learning models are further employed to generate a relit portrait image based on the albedo representation and the shading map.

公开(公告)号：US12079725B2

公开(公告)日：2024-09-03

申请号：US16751897

申请日：2020-01-24

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Yilin Wang ,  Siyuan Qiao ,  Jianming Zhang

IPC: G06N20/00 ,  G06N3/082

CPC classification number: G06N3/082 ,  G06N20/00

Abstract: In some embodiments, an application receives a request to execute a convolutional neural network model. The application determines the computational complexity requirement for the neural network based on the computing resource available on the device. The application further determines the architecture of the convolutional neural network model by determining the locations of down-sampling layers within the convolutional neural network model based on the computational complexity requirement. The application reconfigures the architecture of the convolutional neural network model by moving the down-sampling layers to the determined locations and executes the convolutional neural network model to generate output results.

公开(公告)号：US20240273813A1

公开(公告)日：2024-08-15

申请号：US18168995

申请日：2023-02-14

Applicant: Adobe Inc.

Inventor： Jianming Zhang ,  Yichen Sheng ,  Julien Philip ,  Yannick Hold-Geoffroy ,  Xin Sun ,  He Zhang

IPC: G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/74 ,  G06V10/82

CPC classification number: G06T15/60 ,  G06T7/60 ,  G06V10/60 ,  G06V10/761 ,  G06V10/82

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object shadows for digital images utilizing corresponding geometry-aware buffer channels. For instance, in one or more embodiments, the disclosed systems generate, utilizing a height prediction neural network, an object height map for a digital object portrayed in a digital image and a background height map for a background portrayed in the digital image. The disclosed systems also generate, from the digital image, a plurality of geometry-aware buffer channels using the object height map and the background height map. Further, the disclosed systems modify the digital image to include a soft object shadow for the digital object using the plurality of geometry-aware buffer channels.

公开(公告)号：US20240169623A1

公开(公告)日：2024-05-23

申请号：US18057857

申请日：2022-11-22

Applicant: ADOBE INC.

Inventor： Yu Zeng ,  Zhe Lin ,  Jianming Zhang ,  Qing Liu ,  Jason Wen Yong Kuen ,  John Philip Collomosse

IPC: G06T11/60 ,  G06F40/295 ,  G06T7/11 ,  G06V10/774 ,  G06V10/776

CPC classification number: G06T11/60 ,  G06F40/295 ,  G06T7/11 ,  G06V10/774 ,  G06V10/776 ,  G06T2200/24 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for multi-modal image generation are provided. One or more aspects of the systems and methods includes obtaining a text prompt and layout information indicating a target location for an element of the text prompt within an image to be generated and computing a text feature map including a plurality of values corresponding to the element of the text prompt at pixel locations corresponding to the target location. Then the image is generated based on the text feature map using a diffusion model. The generated image includes the element of the text prompt at the target location.

公开(公告)号：US20240135613A1

公开(公告)日：2024-04-25

申请号：US18320664

申请日：2023-05-19

Applicant: Adobe Inc.

Inventor： Zhihong Ding ,  Scott Cohen ,  Matthew Joss ,  Jianming Zhang ,  Darshan Prasad ,  Celso Gomes ,  Jonathan Brandt

IPC: G06T11/60 ,  G06F3/04842 ,  G06T3/40 ,  G06T5/00 ,  G06T7/50 ,  G06V10/74

CPC classification number: G06T11/60 ,  G06F3/04842 ,  G06T3/40 ,  G06T5/005 ,  G06T7/50 ,  G06V10/761 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that implement perspective-aware object move operations for digital image editing. For instance, in some embodiments, the disclosed systems determine a vanishing point associated with a digital image portraying an object. Additionally, the disclosed systems detect one or more user interactions for moving the object within the digital image. Based on moving the object with respect to the vanishing point, the disclosed systems perform a perspective-based resizing of the object within the digital image.

公开(公告)号：US11853348B2

公开(公告)日：2023-12-26

申请号：US16910440

申请日：2020-06-24

Applicant: Adobe Inc.

Inventor： Akhilesh Kumar ,  Zhe Lin ,  Ratheesh Kalarot ,  Jinrong Xie ,  Jianming Zhang ,  Baldo Antonio Faieta ,  Alex Charles Filipkowski

IPC: G06F16/532 ,  G06F16/583 ,  G06F16/55 ,  G06F16/538 ,  G06N3/02 ,  G06N20/20

CPC classification number: G06F16/532 ,  G06F16/538 ,  G06F16/55 ,  G06F16/583 ,  G06N3/02 ,  G06N20/20

Abstract: Multidimensional digital content search techniques are described that support an ability of a computing device to perform search with increased granularity and flexibility over conventional techniques. In one example, a control is implemented by a computing device that defines a multidimensional (e.g., two-dimensional) continuous space. Locations in the multidimensional continuous space are usable to different search criteria through different weights applied to the criteria associated with the axes. Therefore, user interaction with this control may be used to define a location and corresponding coordinates that may act as weights to the search criteria in order to perform a search of digital content through use of a single user input.

公开(公告)号：US20230360180A1

公开(公告)日：2023-11-09

申请号：US17661985

申请日：2022-05-04

Applicant: Adobe Inc.

Inventor： Haitian Zheng ,  Zhe Lin ,  Jingwan Lu ,  Scott Cohen ,  Elya Shechtman ,  Connelly Barnes ,  Jianming Zhang ,  Ning Xu ,  Sohrab Amirghodsi

IPC: G06V10/40 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T5/005 ,  G06T3/4046 ,  G06V10/40 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing a cascaded modulation inpainting neural network. For example, the disclosed systems utilize a cascaded modulation inpainting neural network that includes cascaded modulation decoder layers. For example, in one or more decoder layers, the disclosed systems start with global code modulation that captures the global-range image structures followed by an additional modulation that refines the global predictions. Accordingly, in one or more implementations, the image inpainting system provides a mechanism to correct distorted local details. Furthermore, in one or more implementations, the image inpainting system leverages fast Fourier convolutions block within different resolution layers of the encoder architecture to expand the receptive field of the encoder and to allow the network encoder to better capture global structure.

公开(公告)号：US11798180B2

公开(公告)日：2023-10-24

申请号：US17186436

申请日：2021-02-26

Applicant: Adobe Inc.

Inventor： Wei Yin ,  Jianming Zhang ,  Oliver Wang ,  Simon Niklaus ,  Mai Long ,  Su Chen

IPC: G06T7/50 ,  G06T7/593 ,  G06T7/30 ,  G06T7/13 ,  G06T7/143 ,  G06T7/521

CPC classification number: G06T7/50 ,  G06T7/13 ,  G06T7/143 ,  G06T7/30 ,  G06T7/521 ,  G06T7/593 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: This disclosure describes one or more implementations of a depth prediction system that generates accurate depth images from single input digital images. In one or more implementations, the depth prediction system enforces different sets of loss functions across mix-data sources to generate a multi-branch architecture depth prediction model. For instance, in one or more implementations, the depth prediction model utilizes different data sources having different granularities of ground truth depth data to robustly train a depth prediction model. Further, given the different ground truth depth data granularities from the different data sources, the depth prediction model enforces different combinations of loss functions including an image-level normalized regression loss function and/or a pair-wise normal loss among other loss functions.

公开(公告)号：US20230325996A1

公开(公告)日：2023-10-12

申请号：US18167690

申请日：2023-02-10

Applicant: Adobe Inc.

Inventor： Zhifei Zhang ,  Jianming Zhang ,  Scott Cohen ,  Zhe Lin

IPC: G06T5/50 ,  G06T3/40 ,  G06V10/60 ,  G06F3/04842

CPC classification number: G06T5/50 ,  G06T3/40 ,  G06V10/60 ,  G06F3/04842 ,  G06T2207/20101 ,  G06T2207/20104 ,  G06T2207/20221

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generates composite images via auto-compositing features. For example, in one or more embodiments, the disclosed systems determine a background image and a foreground object image for use in generating a composite image. The disclosed systems further provide, for display within a graphical user interface of a client device, at least one selectable option for executing an auto-composite model for the composite image, the auto-composite model comprising at least one of a scale prediction model, a harmonization model, or a shadow generation model. The disclosed systems detect, via the graphical user interface, a user selection of the at least one selectable option and generate, in response to detecting the user selection, the composite image by executing the auto-composite model using the background image and the foreground object image.