公开(公告)号：GB2631163A

公开(公告)日：2024-12-25

申请号：GB202407721

申请日：2024-05-31

Applicant: ADOBE INC

Inventor： ZHIFEI ZHANG ,  ZHE LIN ,  YUQIAN ZHOU ,  SOHRAB AMIRGHODSI ,  QING LIU ,  KRISHNA KUMAR SINGH ,  JINGWAN LU ,  ELYA SHECHTMAN

IPC: G06T5/77 ,  G06N3/0475 ,  G06T11/00

Abstract: This application discloses a method comprising: obtaining an input image, a reference image, and a text prompt; encoding, using an image encoder, the reference image to obtain an image embedding; encoding, using a text encoder, the text prompt to obtain a text embedding; and generating, using an image generation model, a composite image based on the input image, the image embedding, and the text embedding. The composite image may depict the input image with a region filled with the content or style from the reference image. The image and text embeddings may be used as guidance for the image generation model. A mask may be received from a user. The image may be generated based on the mask. The text prompt may describe an object in the reference image. The reference image may comprise a portion or a style from the input image. The image generation model may be a diffusion model comprising a self-attention layer. The image and text encoders may be multimodal encoders. Also disclosed is an apparatus to implement the method.

公开(公告)号：GB2606836A

公开(公告)日：2022-11-23

申请号：GB202203239

申请日：2022-03-09

Applicant: ADOBE INC

Inventor： SOHRAB AMIRGHODSI ,  LINGZHI ZHANG ,  ZHE LIN ,  CONNELLY BARNES ,  ELYA SHECHTMAN

IPC: G06T5/00 ,  G06T11/60

Abstract: The present disclosure relates to systems, methods, and computer readable media for accurately, efficiently, and flexibly generating modified digital images utilizing a guided inpainting approach that implements a patch match model informed by a deep visual guide. In particular, the disclosed systems can utilize a visual guide algorithm to automatically generate guidance maps to help identify replacement pixels for inpainting regions of digital images utilizing a patch match model. For example, the disclosed systems can generate guidance maps in the form of structure maps, depth maps, or segmentation maps that respectively indicate the structure, depth, or segmentation of different portions of digital images. Additionally, the disclosed systems can implement a patch match model to identify replacement pixels for filling regions of digital images according to the structure, depth, and/or segmentation of the digital images.

公开(公告)号：GB2601220A

公开(公告)日：2022-05-25

申请号：GB202113083

申请日：2021-09-14

Applicant: ADOBE INC

Inventor： ELYA SHECHTMAN ,  WILLIAM PEEBLES ,  RICHARD ZHANG ,  JUN-YAN ZHU ,  ALYOSHA EFROS

IPC: G06T5/00 ,  G06N3/08 ,  G06N20/00 ,  G06T1/40 ,  G06V10/24 ,  G06V10/82

Abstract: A method of training a generator network 1012 to produce synthetic images comprises receiving an unaligned set of real image data at a spatial transformer 1010 and aligning the real image data set and providing at least one image of the set to a discriminator network 1014. Synthetic images are produced by the generator network and provided to the discriminator network. A training manager 1004 is then used to adversarially train the generator network. The training of the generator is performed using the output of the discriminator network, which determines whether the synthetic image(s) resemble the real image(s). The method may further comprise adversarial training of the spatial transformer using the training manager, the generator, and the discriminator, where the generator produces synthetic aligned images for comparison with the real aligned images produced by the spatial transformer. An image congealing system which performs the disclosed method is also disclosed.

公开(公告)号：GB2578354B

公开(公告)日：2021-12-29

申请号：GB201911506

申请日：2019-08-12

Applicant: ADOBE INC

Inventor： GEOFFREY OXHOLM ,  RAMIZ SHEIKH ,  OLIVER WANG ,  MICHAL LUKAC ,  ELYA SHECHTMAN

IPC: G06T5/00 ,  G06T5/50

Abstract: A method of accessing a video having a target region 306, updating the video content in the target region based on confidence-weighted motion estimation for the target region, and presenting the updated video content on a display device. In particular, the video has a scene comprising a first 112a and a second 112b frame, the scene having an annotation identifying a target region to be modified in one or more video frames. A boundary motion for a boundary of the target region in the scene is computed, the boundary including boundary pixels 404a, 405a neighbouring the target region. Confidence values are assigned to the boundary pixels, wherein a confidence value is based on: a difference between a forward and reverse motion with respect to a particular boundary pixel, and/or a texture in a region that includes the particular boundary pixel. A target motion 412 of a target pixel is interpolated from the boundary motion, wherein the confidence value of the boundary pixel controls a contribution of a motion of the boundary pixel to the target motion. The colour of the target pixel is updated to correspond to the interpolated target motion.

公开(公告)号：GB2589389A

公开(公告)日：2021-06-02

申请号：GB202003400

申请日：2020-03-09

Applicant: ADOBE INC

Inventor： SOHRAB AMIRGHODSI ,  ELYA SHECHTMAN ,  DEREK NELSON NOVO

IPC: G06T5/00 ,  G06T7/11

Abstract: Embodiments of the present disclosure provide systems, methods, and computer storage media for automatically synthesizing a content-aware sampling region for a hole-filling algorithm such as content-aware fill. Given a source image and a hole (or other target region to fill), a sampling region can be synthesized by identifying a band of pixels surrounding the hole, clustering these pixels based on one or more characteristics (e.g., colour, x/y coordinates, depth, focus, etc.), passing each of the resulting clusters as foreground pixels to a segmentation algorithm, and unioning the resulting pixels to form the sampling region. The sampling region can be stored in a constraint mask and passed to a hole-filling algorithm such as content-aware fill to synthesize a fill for the hole (or other target region) from patches sampled from the synthesized sampling region.

公开(公告)号：GB2580596A

公开(公告)日：2020-07-29

申请号：GB201900485

申请日：2019-01-14

Applicant: ADOBE INC

Inventor： SOHRAB AMIRGHODSI ,  ELYA SHECHTMAN ,  SARAH JANE STUCKEY

IPC: G06T11/60 ,  G06T5/00

Abstract: Embodiments of the present disclosure provide systems, methods, and computer storage media for automatically synthesizing a content-aware fill using similarity transformed patches. A user interface receives a user-specified hole and a user-specified sampling region, both of which may be stored in a constraint mask. A brush tool can be used to interactively brush the sampling region and modify the constraint mask. The mask is passed to a patch-based synthesizer configured to synthesize the fill using similarity transformed patches sampled from the sampling region. Fill properties such as similarity transform parameters can be set to control the manner in which the fill is synthesized. A live preview can be provided with gradual updates of the synthesized fill prior to completion. Once a fill has been synthesized, the user interface presents the original image, replacing the hole with the synthesized fill.

公开(公告)号：GB2555916B

公开(公告)日：2020-05-27

申请号：GB201713590

申请日：2017-08-24

Applicant: ADOBE INC

Inventor： XUE BAI ,  ELYA SHECHTMAN ,  SYLVAIN PHILIPPE PARIS

IPC: G06T11/60 ,  G06T3/00 ,  G06T11/40 ,  G06T15/50

公开(公告)号：GB2623402B

公开(公告)日：2025-04-23

申请号：GB202311871

申请日：2023-08-02

Applicant: ADOBE INC

Inventor： ZHE LIN ,  HAITIAN ZHENG ,  ELYA SHECHTMAN ,  JIANMING ZHANG ,  JINGWAN LU ,  NING XU ,  QING LIU ,  SCOTT COHEN ,  SOHRAB AMIRGHODSI

IPC: G06T5/77 ,  G06N3/0464 ,  G06N3/094 ,  G06T7/11 ,  G06V10/26 ,  G06V10/82 ,  G06V20/70

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：GB2623162B

公开(公告)日：2025-01-01

申请号：GB202311936

申请日：2023-08-03

Applicant: ADOBE INC

Inventor： ZHE LIN ,  HAITIAN ZHENG ,  ELYA SHECHTMAN ,  JIANMING ZHANG ,  JINGWAN LU ,  NING XU ,  QING LIU ,  SCOTT COHEN ,  SOHRAB AMIRGHODSI

IPC: G06T5/60 ,  G06T5/77

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for panoptically guiding digital image inpainting utilizing a panoptic inpainting neural network. In some embodiments, the disclosed systems utilize a panoptic inpainting neural network to generate an inpainted digital image according to panoptic segmentation map that defines pixel regions corresponding to different panoptic labels. In some cases, the disclosed systems train a neural network utilizing a semantic discriminator that facilitates generation of digital images that are realistic while also conforming to a semantic segmentation. The disclosed systems generate and provide a panoptic inpainting interface to facilitate user interaction for inpainting digital images. In certain embodiments, the disclosed systems iteratively update an inpainted digital image based on changes to a panoptic segmentation map.

公开(公告)号：GB2585396A

公开(公告)日：2021-01-13

申请号：GB201916984

申请日：2019-11-21

Applicant: ADOBE INC

Inventor： WENQI XIAN ,  MATTHEW FISHER ,  JONATHAN EISENMANN ,  GEOFFREY OXHOLM ,  ELYA SHECHTMAN

IPC: G06T7/80 ,  G06N20/00 ,  G06T7/536 ,  G06T7/70

Abstract: Th present disclosure relates to a system, computer-readable media and method for utilizing a critical edge detection neural network and a geometric model to determine camera parameter from a single digital image 802. In particular, in one or more embodiments, the disclosed systems can train and utilize a critical edge detection neural network to generate a vanishing edge map indicating vanishing lines from the digital image 804. The system can then utilize the vanishing edge map to determine camera parameters by applying a geometric model to the vanishing edge map 806. Further, the system can generate ground truth vanishing line data from a set of training digital images for training the critical edge detection neural network (Fig.9, 904). A critical edge detection neural network is trained by determining the distances between a vanishing point and a training line (Fig 9. 906). Predicted vanishing lines are generated to modify and train the neural network. The neural network may be a convolutional neural network (CNN) (Fig.9, 912). The camera parameters may be pitch, yaw, roll or focal length and the edge map may contain weights or confidence values.