-
公开(公告)号:US12056857B2
公开(公告)日:2024-08-06
申请号:US17520249
申请日:2021-11-05
Applicant: Adobe Inc.
Inventor: Yuqian Zhou , Connelly Barnes , Sohrab Amirghodsi , Elya Shechtman
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately generating inpainted digital images utilizing a guided inpainting model guided by both plane panoptic segmentation and plane grouping. For example, the disclosed systems utilize a guided inpainting model to fill holes of missing pixels of a digital image as informed or guided by an appearance guide and a geometric guide. Specifically, the disclosed systems generate an appearance guide utilizing plane panoptic segmentation and generate a geometric guide by grouping plane panoptic segments. In some embodiments, the disclosed systems generate a modified digital image by implementing an inpainting model guided by both the appearance guide (e.g., a plane panoptic segmentation map) and the geometric guide (e.g., a plane grouping map).
-
公开(公告)号:US11854244B2
公开(公告)日:2023-12-26
申请号:US18048311
申请日:2022-10-20
Applicant: Adobe Inc.
Inventor: Sohrab Amirghodsi , Zhe Lin , Yilin Wang , Tianshu Yu , Connelly Barnes , Elya Shechtman
IPC: G06V10/75 , G06F17/18 , G06N3/08 , G06N20/00 , G06V10/82 , G06F18/214 , G06F18/22 , G06F18/211 , G06F18/213 , G06V10/74 , G06V10/771 , G06V10/774 , G06V20/70
CPC classification number: G06V10/757 , G06F17/18 , G06F18/211 , G06F18/213 , G06F18/214 , G06F18/22 , G06N3/08 , G06N20/00 , G06V10/761 , G06V10/771 , G06V10/774 , G06V10/82 , G06V20/70
Abstract: A panoptic labeling system includes a modified panoptic labeling neural network (“modified PLNN”) that is trained to generate labels for pixels in an input image. The panoptic labeling system generates modified training images by combining training images with mask instances from annotated images. The modified PLNN determines a set of labels representing categories of objects depicted in the modified training images. The modified PLNN also determines a subset of the labels representing categories of objects depicted in the input image. For each mask pixel in a modified training image, the modified PLNN calculates a probability indicating whether the mask pixel has the same label as an object pixel. The modified PLNN generates a mask label for each mask pixel, based on the probability. The panoptic labeling system provides the mask label to, for example, a digital graphics editing system that uses the labels to complete an infill operation.
-
13.
公开(公告)号:US20230368339A1
公开(公告)日:2023-11-16
申请号:US17663317
申请日:2022-05-13
Applicant: Adobe Inc.
Inventor: Haitian Zheng , Zhe Lin , Jingwan Lu , Scott Cohen , Elya Shechtman , Connelly Barnes , Jianming Zhang , Ning Xu , Sohrab Amirghodsi
CPC classification number: G06T5/005 , G06T7/11 , G06N3/04 , G06T2207/20081 , G06T2207/20084
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that generate inpainted digital images utilizing class-specific cascaded modulation inpainting neural network. For example, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network that includes cascaded modulation decoder layers to generate replacement pixels portraying a particular target object class. To illustrate, in response to user selection of a replacement region and target object class, the disclosed systems utilize a class-specific cascaded modulation inpainting neural network corresponding to the target object class to generate an inpainted digital image that portrays an instance of the target object class within the replacement region. Moreover, in one or more embodiments the disclosed systems train class-specific cascaded modulation inpainting neural networks corresponding to a variety of target object classes, such as a sky object class, a water object class, a ground object class, or a human object class.
-
公开(公告)号:US11507777B2
公开(公告)日:2022-11-22
申请号:US15930539
申请日:2020-05-13
Applicant: Adobe Inc.
Inventor: Sohrab Amirghodsi , Zhe Lin , Yilin Wang , Tianshu Yu , Connelly Barnes , Elya Shechtman
Abstract: A panoptic labeling system includes a modified panoptic labeling neural network (“modified PLNN”) that is trained to generate labels for pixels in an input image. The panoptic labeling system generates modified training images by combining training images with mask instances from annotated images. The modified PLNN determines a set of labels representing categories of objects depicted in the modified training images. The modified PLNN also determines a subset of the labels representing categories of objects depicted in the input image. For each mask pixel in a modified training image, the modified PLNN calculates a probability indicating whether the mask pixel has the same label as an object pixel. The modified PLNN generates a mask label for each mask pixel, based on the probability. The panoptic labeling system provides the mask label to, for example, a digital graphics editing system that uses the labels to complete an infill operation.
-
公开(公告)号:US20200342634A1
公开(公告)日:2020-10-29
申请号:US16392968
申请日:2019-04-24
Applicant: Adobe Inc.
Inventor: Connelly Barnes , Sohrab Amirghodsi , Michal Lukac , Elya Shechtman , Ning Yu
Abstract: Techniques are disclosed for neural network based interpolation of image textures. A methodology implementing the techniques according to an embodiment includes training a global encoder network to generate global latent vectors based on training texture images, and training a local encoder network to generate local latent tensors based on the training texture images. The method further includes interpolating between the global latent vectors associated with each set of training images, and interpolating between the local latent tensors associated with each set of training images. The method further includes training a decoder network to generate reconstructions of the training texture images and to generate an interpolated texture based on the interpolated global latent vectors and the interpolated local latent tensors. The training of the encoder and decoder networks is based on a minimization of a loss function of the reconstructions and a minimization of a loss function of the interpolated texture.
-
公开(公告)号:US10762680B1
公开(公告)日:2020-09-01
申请号:US16363839
申请日:2019-03-25
Applicant: Adobe Inc.
Inventor: Sohrab Amirghodsi , Connelly Barnes , Eric L. Palmer
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating deterministic enhanced digital images based on parallel determinations of pixel group offsets arranged in pixel waves. For example, the disclosed systems can utilize a parallel wave analysis to propagate through pixel groups in a pixel wave of a target region within a digital image to determine matching patch offsets for the pixel groups. The disclosed systems can further utilize the matching patch offsets to generate a deterministic enhanced digital image by filling or replacing pixels of the target region with matching pixels indicated by the matching patch offsets.
-
公开(公告)号:US12217169B2
公开(公告)日:2025-02-04
申请号:US17198670
申请日:2021-03-11
Applicant: ADOBE INC.
Inventor: Ishit bhadresh Mehta , Michaël Gharbi , Connelly Barnes , Elya Shechtman
IPC: G06N3/04 , G06N3/0455 , G06N3/048 , G06N3/08 , G06T3/4007 , G06T5/77
Abstract: Systems and methods for signal processing are described. Embodiments receive a digital signal comprising original signal values corresponding to a discrete set of original sample locations, generate modulation parameters based on the digital signal using a modulator network, wherein each of a plurality of modulator layers of the modulator network outputs a set of the modulation parameters, and generate a predicted signal value of the digital signal at an additional location using a synthesizer network, wherein each of a plurality of synthesizer layers of the synthesizer network operates based on the set of the modulation parameters from a corresponding modulator layer of the modulator network.
-
公开(公告)号:US20240362791A1
公开(公告)日:2024-10-31
申请号:US18307353
申请日:2023-04-26
Applicant: Adobe Inc.
Inventor: Yuqian Zhou , Chuong Huynh , Connelly Barnes , Elya Shechtman , Sohrab Amirghodsi , Zhe Lin
IPC: G06T7/12 , G06F3/04883 , G06V10/44 , G06V10/74 , G06V10/80
CPC classification number: G06T7/12 , G06F3/04883 , G06V10/44 , G06V10/761 , G06V10/806 , G06T2207/20101 , G06V2201/07
Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning to generate a mask for an object portrayed in a digital image. For example, in some embodiments, the disclosed systems utilize a neural network to generate an image feature representation from the digital image. The disclosed systems can receive a selection input identifying one or more pixels corresponding to the object. In addition, in some implementations, the disclosed systems generate a modified feature representation by integrating the selection input into the image feature representation. Moreover, in one or more embodiments, the disclosed systems utilize an additional neural network to generate a plurality of masking proposals for the object from the modified feature representation. Furthermore, in some embodiments, the disclosed systems utilize a further neural network to generate the mask for the object from the modified feature representation and/or the masking proposals.
-
公开(公告)号:US20240303787A1
公开(公告)日:2024-09-12
申请号:US18179855
申请日:2023-03-07
Applicant: Adobe Inc.
Inventor: Yuqian Zhou , Connelly Barnes , Zijun Wei , Zhe Lin , Elya Shechtman , Sohrab Amirghodsi , Xiaoyang Liu
CPC classification number: G06T5/77 , G06T7/11 , G06V20/176 , G06T2207/20021 , G06T2207/30184
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for inpainting a digital image using a hybrid wire removal pipeline. For example, the disclosed systems use a hybrid wire removal pipeline that integrates multiple machine learning models, such as a wire segmentation model, a hole separation model, a mask dilation model, a patch-based inpainting model, and a deep inpainting model. Using the hybrid wire removal pipeline, in some embodiments, the disclosed systems generate a wire segmentation from a digital image depicting one or more wires. The disclosed systems also utilize the hybrid wire removal pipeline to extract or identify portions of the wire segmentation that indicate specific wires or portions of wires. In certain embodiments, the disclosed systems further inpaint pixels of the digital image corresponding to the wires indicated by the wire segmentation mask using the patch-based inpainting model and/or the deep inpainting model.
-
20.
公开(公告)号:US20230385992A1
公开(公告)日:2023-11-30
申请号:US17664991
申请日:2022-05-25
Applicant: Adobe Inc.
Inventor: Connelly Barnes , Elya Shechtman , Sohrab Amirghodsi , Zhe Lin
CPC classification number: G06T5/005 , G06T5/50 , G06T2207/20084 , G06T2207/20212 , G06T2207/10024
Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media that implement an inpainting framework having computer-implemented machine learning models to generate high-resolution inpainting results. For instance, in one or more embodiments, the disclosed systems generate an inpainted digital image utilizing a deep inpainting neural network from a digital image having a replacement region. The disclosed systems further generate, utilizing a visual guide algorithm, at least one deep visual guide from the inpainted digital image. Using a patch match model and the at least one deep visual guide, the disclosed systems generate a plurality of modified digital images from the digital image by replacing the region of pixels of the digital image with replacement pixels. Additionally, the disclosed systems select, utilizing an inpainting curation model, a modified digital image from the plurality of modified digital images to provide to a client device.
-
-
-
-
-
-
-
-
-