公开(公告)号：US20230058793A1

公开(公告)日：2023-02-23

申请号：US18045730

申请日：2022-10-11

Applicant: Adobe Inc.

Inventor： Federico Perazzi ,  Jingwan Lu

IPC: G06T5/00 ,  G06T7/11 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06T3/40 ,  G06V30/262

Abstract: The present disclosure relates to an image retouching system that automatically retouches digital images by accurately correcting face imperfections such as skin blemishes and redness. For instance, the image retouching system automatically retouches a digital image through separating digital images into multiple frequency layers, utilizing a separate corresponding neural network to apply frequency-specific corrections at various frequency layers, and combining the retouched frequency layers into a retouched digital image. As described herein, the image retouching system efficiently utilizes different neural networks to target and correct skin features specific to each frequency layer.

公开(公告)号：US20220122305A1

公开(公告)日：2022-04-21

申请号：US17384273

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T11/60 ,  G06T3/40

Abstract: An improved system architecture uses a pipeline including an encoder and a Generative Adversarial Network (GAN) including a generator neural network to generate edited images with improved speed, realism, and identity preservation. The encoder produces an initial latent space representation of an input image by encoding the input image. The generator neural network generates an initial output image by processing the initial latent space representation of the input image. The system generates an optimized latent space representation of the input image using a loss minimization technique that minimizes a loss between the input image and the initial output image. The loss is based on target perceptual features extracted from the input image and initial perceptual features extracted from the initial output image. The system outputs the optimized latent space representation of the input image for downstream use.

公开(公告)号：US20220122222A1

公开(公告)日：2022-04-21

申请号：US17384283

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06T3/40 ,  G06T11/60

Abstract: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.

公开(公告)号：US20220121932A1

公开(公告)日：2022-04-21

申请号：US17384378

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Ratheesh Kalarot ,  Wei-An Lin ,  Cameron Smith ,  Zhixin Shu ,  Baldo Faieta ,  Shabnam Ghadar ,  Jingwan Lu ,  Aliakbar Darabi ,  Jun-Yan Zhu ,  Niloy Mitra ,  Richard Zhang ,  Elya Shechtman

IPC: G06N3/08 ,  G06N3/04

Abstract: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.

公开(公告)号：US11232607B2

公开(公告)日：2022-01-25

申请号：US16751959

申请日：2020-01-24

Applicant: Adobe Inc.

Inventor： Nishant Kumar ,  Vikas Sharma ,  Shantanu Agarwal ,  Sameer Bhatt ,  Rupali Arora ,  Richard Zhang ,  Anuradha Yadav ,  Jingwan Lu ,  Matthew David Fisher

IPC: G06T11/00 ,  G06T7/90 ,  G06K9/46 ,  H04N1/60

Abstract: In implementations of adding color to digital images, an image colorization system can display a digital image to be color adjusted in an image editing interface and convert pixel content of the digital image to a LAB color space. The image colorization system can determine a lightness value (L) in the LAB color space of the pixel content of the digital image at a specified point on the digital image, and determine colors representable in an RGB color space based on combinations of A,B value pairs with the lightness value (L) in the LAB color space. The image colorization system can then determine a range of the colors for display in a color gamut in the image editing interface, the range of the colors corresponding to the A,B value pairs with the lightness value (L) of the pixel content at the specified point on the digital image.

公开(公告)号：US20210319532A1

公开(公告)日：2021-10-14

申请号：US16848741

申请日：2020-04-14

Applicant: Adobe Inc.

Inventor： Julia Gong ,  Yannick Hold-Geoffroy ,  Jingwan Lu

IPC: G06T3/00 ,  G06T7/00 ,  G06T11/60

Abstract: Techniques and systems are provided for configuring neural networks to perform warping of an object represented in an image to create a caricature of the object. For instance, in response to obtaining an image of an object, a warped image generator generates a warping field using the image as input. The warping field is generated using a model trained with pairings of training images and known warped images using supervised learning techniques and one or more losses. The warped image generator determines, based on the warping field, a set of displacements associated with pixels of the input image. These displacements indicate pixel displacement directions for the pixels of the input image. These displacements are applied to the digital image to generate a warped image of the object.

公开(公告)号：US11093660B2

公开(公告)日：2021-08-17

申请号：US14968870

申请日：2015-12-14

Applicant: Adobe Inc.

Inventor： Paul Asente ,  Jingwan Lu ,  Huy Phan

IPC: G06F30/00 ,  G06N7/00

Abstract: Methods and systems for aiding users in generating object pattern designs with increased speed. In particular, one or more embodiments train a sequence-based machine-learning model using training objects, each training object including a plurality of regions with a plurality of design elements. One or more embodiments identify a plurality regions of an object with a first region adjacent a second region. One or more embodiments receive a user selection of a design element for populating the first region with a first design element from a plurality of design elements. One or more embodiments identify a second design element from the plurality of design elements based on the first design element using the trained sequence-based machine-learning model. One or more embodiments also populate the second region with one or more instances of the second design element.

公开(公告)号：US11048335B2

公开(公告)日：2021-06-29

申请号：US16846895

申请日：2020-04-13

Applicant: Adobe Inc.

Inventor： Jingwan Lu ,  Stephen J. DiVerdi ,  Byungmoon Kim ,  Jun Xing

IPC: G06T19/00 ,  G06F3/01 ,  G06T17/30 ,  G06F3/0346 ,  G06F3/0488 ,  G06T11/20 ,  G06T15/02

Abstract: Stroke operation prediction techniques and systems for three-dimensional digital content are described. In one example, stroke operation data is received that describes a stroke operation input via a user interface as part of the three-dimensional digital content. A cycle is generated that defines a closed path within the three-dimensional digital content based on the input stroke operation and at least one other stroke operation in the user interface. A surface is constructed based on the generated cycle. A predicted stroke operation is generated based at least in part on the constructed surface. The predicted stroke operation is then output in real time in the user interface as part of the three-dimensional digital content as the stroke operation data is received.

公开(公告)号：US20210158494A1

公开(公告)日：2021-05-27

申请号：US17170209

申请日：2021-02-08

Applicant: Adobe Inc.

Inventor： Jingwan Lu ,  Patsorn Sangkloy ,  Chen Fang

IPC: G06T5/20 ,  G06T11/00 ,  G06N3/04 ,  G06N3/08

Abstract: Methods and systems are provided for transforming sketches into stylized electronic paintings. A neural network system is trained where the training includes training a first neural network that converts input sketches into output images and training a second neural network that converts images into output paintings. Similarity for the first neural network is evaluated between the output image and a reference image and similarity for the second neural network is evaluated between the output painting, the output image, and a reference painting. The neural network system is modified based on the evaluated similarity. The trained neural network is used to generate an output painting from an input sketch where the output painting maintains features from the input sketch utilizing an extrapolated intermediate image and reflects a designated style from the reference painting.

公开(公告)号：US10997752B1

公开(公告)日：2021-05-04

申请号：US16813050

申请日：2020-03-09

Applicant: Adobe Inc.

Inventor： Seungjoo Yoo ,  Richard Zhang ,  Matthew Fisher ,  Jingwan Lu

IPC: G06T7/13 ,  G06T11/00 ,  G06T7/90

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing an edge prediction neural network and edge-guided colorization neural network to transform grayscale digital images into colorized digital images. In one or more embodiments, the disclosed systems apply a color edge prediction neural network to a grayscale image to generate a color edge map indicating predicted chrominance edges. The disclosed systems can present the color edge map to a user via a colorization graphical user interface and receive user color points and color edge modifications. The disclosed systems can apply a second neural network, an edge-guided colorization neural network, to the color edge map or a modified edge map, user color points, and the grayscale image to generate an edge-constrained colorized digital image.