公开(公告)号：US20230143721A1

公开(公告)日：2023-05-11

申请号：US17524282

申请日：2021-11-11

Applicant: ADOBE INC.

Inventor： Sungchul Kim ,  Subrata Mitra ,  Ruiyi Zhang ,  Rui Wang ,  Handong Zhao ,  Tong Yu

IPC: G06F40/295 ,  G06N20/00

CPC classification number: G06F40/295 ,  G06N20/00

Abstract: Embodiments of the technology described herein describe a machine classifier capable of continually learning new classes through a continual few-shot learning approach. A natural language processing (NLP) machine classifier may initially be trained to identify a plurality of other classes through a conventional training process. In order to learn a new class, natural-language training data for a new class is generated. The training data for the new class may be few-shot training data. The training also uses synthetic training data that represents each of the plurality of other classes. The synthetic training data may be generated through a model inversion of the original classifier. The synthetic training data and the natural-language training data are used to retrain the NLP classifier to identify text in the plurality of other classes and the new class using.

公开(公告)号：US20230133522A1

公开(公告)日：2023-05-04

申请号：US17513127

申请日：2021-10-28

Applicant: Adobe Inc.

Inventor： Handong Zhao ,  Zhe Lin ,  Zhaowen Wang ,  Zhankui He ,  Ajinkya Gorakhnath Kale

IPC: G06F16/245 ,  G06F16/248 ,  G06N20/00

Abstract: Digital content search techniques are described that overcome the challenges found in conventional sequence-based techniques through use of a query-aware sequential search. In one example, a search query is received and sequence input data is obtained based on the search query. The sequence input data describes a sequence of digital content and respective search queries. Embedding data is generated based on the sequence input data using an embedding module of a machine-learning model. The embedding module includes a query-aware embedding layer that generates embeddings of the sequence of digital content and respective search queries. A search result is generated referencing at least one item of digital content by processing the embedding data using at least one layer of the machine-learning model.

公开(公告)号：US20230116969A1

公开(公告)日：2023-04-20

申请号：US17501191

申请日：2021-10-14

Applicant: Adobe Inc.

Inventor： Handong Zhao ,  Zhankui He ,  Zhaowen Wang ,  Ajinkya Gorakhnath Kale ,  Zhe Lin

IPC: G06F16/438 ,  G06F16/44 ,  G06N3/04

Abstract: Digital content search techniques are described. In one example, the techniques are incorporated as part of a multi-head self-attention module of a transformer using machine learning. A localized self-attention module, for instance, is incorporated as part of the multi-head self-attention module that applies local constraints to the sequence. This is performable in a variety of ways. In a first instance, a model-based local encoder is used, examples of which include a fixed-depth recurrent neural network (RNN) and a convolutional network. In a second instance, a masking-based local encoder is used, examples of which include use of a fixed window, Gaussian initialization, and an adaptive predictor.

公开(公告)号：US11574142B2

公开(公告)日：2023-02-07

申请号：US16943511

申请日：2020-07-30

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Hung Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06K9/62 ,  G06T11/00 ,  G06V10/44

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

公开(公告)号：US20220398230A1

公开(公告)日：2022-12-15

申请号：US17347133

申请日：2021-06-14

Applicant: Adobe Inc.

Inventor： Ritwik Sinha ,  Saayan Mitra ,  Handong Zhao ,  Somdeb Sarkhel ,  Trevor Paulsen ,  William Brandon George

IPC: G06F16/215 ,  G06F16/242 ,  G06N5/04 ,  G06N20/00

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating automatic suggestions to effectively modify the organization of an ingested data collection without destruction of the underlying raw data. In particular, in one or more embodiments, the disclosed systems utilize multiple machine learning models in sequence to determine likelihoods that the organizational structure of an ingested data collection should be modified in various ways. In response to generating these likelihoods, the disclosed systems generate corresponding automatic suggestions to modify the organization of the ingested data collection. In response to a detected selection of one or more of the automatic suggestions, the disclosed systems read data out of the ingested data collection in accordance with the selected automatic suggestions to effectively modify the organization of the ingested data collection.

公开(公告)号：US20220244815A1

公开(公告)日：2022-08-04

申请号：US17161770

申请日：2021-01-29

Applicant: Adobe Inc.

Inventor： Camille Harris ,  Zening Qu ,  Sana Lee ,  Ryan Rossi ,  Fan Du ,  Eunyee Koh ,  Tak Yeon Lee ,  Sungchul Kim ,  Handong Zhao ,  Sumit Shekhar

IPC: G06F3/0482 ,  G06F3/0484 ,  G06F17/15

Abstract: In some embodiments, a data visualization system detects insights from a dataset and computes insight scores for respective insights. The data visualization system further computes insight type scores, from the insight scores, for insight types in the detected insights. The data visualization system determines a selected insight type for the dataset having a higher insight type score than unselected insight types and determines, for the selected insight type, a set of selected insights that have higher insight scores than unselected insights. The data visualization system determines insight visualizations for the set of selected insights and generates, for inclusion in a user interface of the data visualization system, selectable interface elements configured for invoking an editing tool for updating the determined insight visualizations from the dataset. The selectable interface elements are arranged in the user interface according to the insight scores of the set of selected insights.

公开(公告)号：US20220147838A1

公开(公告)日：2022-05-12

申请号：US17093185

申请日：2020-11-09

Applicant: Adobe Inc.

Inventor： Jiuxiang Gu ,  Vlad Ion Morariu ,  Tong Sun ,  Jason wen yong Kuen ,  Handong Zhao

IPC: G06N5/02 ,  G06N7/00 ,  G06T7/90 ,  G06N3/08

Abstract: Methods and systems disclosed herein relate generally to systems and methods for generating visual relationship graphs that identify relationships between objects depicted in an image. A vision-language application uses transformer encoders to generate a graph structure, in which the graph structure represents a dependency between a first region and a second region of an image. The dependency indicates that a contextual representation of the first region was derived, at least in part, by processing the second region. The contextual representation identifies a predicted identity of an image object depicted in the first region. The predicted identity is determined at least in part by identifying a relationship between the first region and other data objects associated with various modalities.

公开(公告)号：US20210319056A1

公开(公告)日：2021-10-14

申请号：US16843218

申请日：2020-04-08

Applicant: ADOBE INC.

Inventor： Handong Zhao ,  Ajinkya Kale ,  Xiaowei Jia ,  Zhe Lin

IPC: G06F16/43 ,  G06F16/45 ,  G06F16/438 ,  G06N3/04 ,  G06N3/08

Abstract: The present disclosure relates to a retrieval method including: generating a graph representing a set of users, items, and queries; generating clusters from the media items; generating embeddings for each cluster from embeddings of the items within the corresponding cluster; generating augmented query embeddings for each cluster from the embedding of the corresponding cluster and query embeddings of the queries; inputting the cluster embeddings and the augmented query embeddings to a layer of a graph convolutional network (GCN) to determine user embeddings of the users; inputting the embedding of the given user and a query embedding of the given query to a layer of the GCN to determine a user-specific query embedding; generating a score for each of the items based on the item embeddings and the user-specific query embedding; and presenting the items having the score exceeding a threshold.

公开(公告)号：US20210312232A1

公开(公告)日：2021-10-07

申请号：US16885168

申请日：2020-05-27

Applicant: Adobe Inc.

Inventor： Christopher Tensmeyer ,  Vlad Ion Morariu ,  Varun Manjunatha ,  Tong Sun ,  Nikolaos Barmpalios ,  Kai Li ,  Handong Zhao ,  Curtis Wigington

IPC: G06K9/62 ,  G06N5/04 ,  G06K9/00

Abstract: A domain alignment technique for cross-domain object detection tasks is introduced. During a preliminary pretraining phase, an object detection model is pretrained to detect objects in images associated with a source domain using a source dataset of images associated with the source domain. After completing the pretraining phase, a domain adaptation phase is performed using the source dataset and a target dataset to adapt the pretrained object detection model to detect objects in images associated with the target domain. The domain adaptation phase may involve the use of various domain alignment modules that, for example, perform multi-scale pixel/path alignment based on input feature maps or perform instance-level alignment based on input region proposals.

公开(公告)号：US20210271707A1

公开(公告)日：2021-09-02

申请号：US16803480

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06F16/583 ,  G06K9/62 ,  G06K9/72 ,  G06F40/30 ,  G06F16/538 ,  G06F16/56 ,  G06F16/2457 ,  G06N3/08

Abstract: Certain embodiments involve a method for generating a search result. The method includes processing devices performing operations including receiving a query having a text input by a joint embedding model trained to generate an image result. Training the joint embedding model includes accessing a set of images and textual information. Training further includes encoding the images into image feature vectors based on spatial features. Further, training includes encoding the textual information into textual feature vectors based on semantic information. Training further includes generating a set of image-text pairs based on matches between image feature vectors and textual feature vectors. Further, training includes generating a visual grounding dataset based on spatial information. Training further includes generating a set of visual-semantic joint embeddings by grounding the image-text pairs with the visual grounding dataset. Additionally, operations include generating an image result for display by the joint embedding model based on the text input.