公开(公告)号：US20230401718A1

公开(公告)日：2023-12-14

申请号：US17838995

申请日：2022-06-13

Applicant: ADOBE INC.

Inventor： Zijun Wei ,  Yilin Wang ,  Jianming Zhang ,  He Zhang

IPC: G06T7/11 ,  G06T7/136 ,  G06V10/46 ,  G06V10/764

CPC classification number: G06T7/11 ,  G06T7/136 ,  G06V10/462 ,  G06V10/764

Abstract: An image processing system generates an image mask from an image. The image is processed by an object detector to identify a region having an object, and the region is classified based on an object type of the object. A masking pipeline is selected from a number of masking pipelines based on the classification of the region. The region is processed using the masking pipeline to generate a region mask. An image mask for the image is generated using the region mask.

公开(公告)号：US20230401717A1

公开(公告)日：2023-12-14

申请号：US17806314

申请日：2022-06-10

Applicant: ADOBE INC.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zijun Wei ,  Zhe Lin

IPC: G06T7/11 ,  G06V20/70 ,  G06V10/778 ,  G06V10/82

CPC classification number: G06T7/11 ,  G06V20/70 ,  G06V10/778 ,  G06V10/82 ,  G06T2207/20021 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods for image segmentation are described. Embodiments of the present disclosure receive an image depicting an object; generate image features for the image by performing an atrous self-attention operation based on a plurality of dilation rates for a convolutional kernel applied at a position of a sliding window on the image; and generate label data that identifies the object based on the image features.

公开(公告)号：US20230206462A1

公开(公告)日：2023-06-29

申请号：US18175481

申请日：2023-02-27

Applicant: Adobe Inc.

Inventor： Qihang Yu ,  Jianming Zhang ,  He Zhang ,  Yilin Wang ,  Zhe Lin ,  Ning Xu

IPC: G06T7/194 ,  G06T7/136 ,  G06T7/11 ,  G06T3/40 ,  G06T5/00

CPC classification number: G06T7/194 ,  G06T7/136 ,  G06T7/11 ,  G06T3/4053 ,  G06T3/4046 ,  G06T5/009 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a progressive refinement network to refine alpha mattes generated utilizing a mask-guided matting neural network. In particular, the disclosed systems can use the matting neural network to process a digital image and a coarse guidance mask to generate alpha mattes at discrete neural network layers. In turn, the disclosed systems can use the progressive refinement network to combine alpha mattes and refine areas of uncertainty. For example, the progressive refinement network can combine a core alpha matte corresponding to more certain core regions of a first alpha matte and a boundary alpha matte corresponding to uncertain boundary regions of a second, higher resolution alpha matte. Based on the combination of the core alpha matte and the boundary alpha matte, the disclosed systems can generate a final alpha matte for use in image matting processes.

公开(公告)号：US20230123658A1

公开(公告)日：2023-04-20

申请号：US17502782

申请日：2021-10-15

Applicant: Adobe Inc.

Inventor： Yifan Liu ,  Jianming Zhang ,  He Zhang ,  Elya Shechtman ,  Zhe Lin

IPC: G06T15/60 ,  G06T11/00 ,  G06T7/70 ,  G06T7/194 ,  G06N3/04

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate a height map for a digital object portrayed in a digital image and further utilizes the height map to generate a shadow for the digital object. Indeed, in one or more embodiments, the disclosed systems generate (e.g., utilizing a neural network) a height map that indicates the pixels heights for pixels of a digital object portrayed in a digital image. The disclosed systems utilize the pixel heights, along with lighting information for the digital image, to determine how the pixels of the digital image project to create a shadow for the digital object. Further, in some implementations, the disclosed systems utilize the determined shadow projections to generate (e.g., utilizing another neural network) a soft shadow for the digital object. Accordingly, in some cases, the disclosed systems modify the digital image to include the shadow.

公开(公告)号：US20220292684A1

公开(公告)日：2022-09-15

申请号：US17200525

申请日：2021-03-12

Applicant: Adobe Inc.

Inventor： Yilin Wang ,  Chenglin Yang ,  Jianming Zhang ,  He Zhang ,  Zhe Lin

IPC: G06T7/11 ,  G06K9/62 ,  G06T3/40 ,  G06N3/04 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilizes a neural network having a hierarchy of hierarchical point-wise refining blocks to generate refined segmentation masks for high-resolution digital visual media items. For example, in one or more embodiments, the disclosed systems utilize a segmentation refinement neural network having an encoder and a recursive decoder to generate the refined segmentation masks. The recursive decoder includes a deconvolution branch for generating feature maps and a refinement branch for generating and refining segmentation masks. In particular, in some cases, the refinement branch includes a hierarchy of hierarchical point-wise refining blocks that recursively refine a segmentation mask generated for a digital visual media item. In some cases, the disclosed systems utilize a segmentation refinement neural network that includes a low-resolution network and a high-resolution network, each including an encoder and a recursive decoder, to generate the refined segmentation masks.

公开(公告)号：US20220262009A1

公开(公告)日：2022-08-18

申请号：US17177595

申请日：2021-02-17

Applicant: Adobe Inc.

Inventor： Qihang Yu ,  Jianming Zhang ,  He Zhang ,  Yilin Wang ,  Zhe Lin ,  Ning Xu

IPC: G06T7/194 ,  G06T7/136 ,  G06T7/11 ,  G06T5/00 ,  G06T3/40

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a progressive refinement network to refine alpha mattes generated utilizing a mask-guided matting neural network. In particular, the disclosed systems can use the matting neural network to process a digital image and a coarse guidance mask to generate alpha mattes at discrete neural network layers. In turn, the disclosed systems can use the progressive refinement network to combine alpha mattes and refine areas of uncertainty. For example, the progressive refinement network can combine a core alpha matte corresponding to more certain core regions of a first alpha matte and a boundary alpha matte corresponding to uncertain boundary regions of a second, higher resolution alpha matte. Based on the combination of the core alpha matte and the boundary alpha matte, the disclosed systems can generate a final alpha matte for use in image matting processes.

公开(公告)号：US11158055B2

公开(公告)日：2021-10-26

申请号：US16523465

申请日：2019-07-26

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Jianming Zhang ,  He Zhang ,  Federico Perazzi

IPC: G06K9/46 ,  G06T7/10 ,  G06N3/04 ,  G06N3/08 ,  G06T11/60

Abstract: The present disclosure relates to utilizing a neural network having a two-stream encoder architecture to accurately generate composite digital images that realistically portray a foreground object from one digital image against a scene from another digital image. For example, the disclosed systems can utilize a foreground encoder of the neural network to identify features from a foreground image and further utilize a background encoder to identify features from a background image. The disclosed systems can then utilize a decoder to fuse the features together and generate a composite digital image. The disclosed systems can train the neural network utilizing an easy-to-hard data augmentation scheme implemented via self-teaching. The disclosed systems can further incorporate the neural network within an end-to-end framework for automation of the image composition process.