公开(公告)号：US12288074B2

公开(公告)日：2025-04-29

申请号：US16261092

申请日：2019-01-29

Applicant: Adobe Inc.

Inventor： Yash Chandak ,  Georgios Theocharous

IPC: G06F9/38 ,  G06F9/48 ,  G06N3/08 ,  G06N20/00

Abstract: The present disclosure relates to generating proposed digital actions in high-dimensional action spaces for client devices utilizing reinforcement learning models. For example, the disclosed systems can utilize a supervised machine learning model to train a latent representation decoder to determine proposed digital actions based on latent representations. Additionally, the disclosed systems can utilize a latent representation policy gradient model to train a state-based latent representation generation policy to generate latent representations based on the current state of client devices. Subsequently, the disclosed systems can identify the current state of a client device and a plurality of available actions, utilize the state-based latent representation generation policy to generate a latent representation based on the current state, and utilize the latent representation decoder to determine a proposed digital action from the plurality of available actions by analyzing the latent representation.

公开(公告)号：US12047273B2

公开(公告)日：2024-07-23

申请号：US17671075

申请日：2022-02-14

Applicant: ADOBE INC.

Inventor： Georgios Theocharous ,  Kai Wang ,  Zhao Song ,  Sridhar Mahadevan

IPC: H04L45/02 ,  H04L41/147

CPC classification number: H04L45/08 ,  H04L41/147

Abstract: A control system facilitates active management of a streaming data system. Given historical data traffic for each data stream processed by a streaming data system, the control system uses a machine learning model to predict future data traffic for each data stream. The control system selects a matching between data streams and servers for a future time that minimizes a cost comprising a switching cost and a server imbalance cost based on the predicted data traffic for the future time. In some configurations, the matching is selected using a planning window comprising a number of future time steps dynamically selected based on uncertainty associated with the predicted data traffic. Given the selected matching, the control system may manage the streaming data system by causing data streams to be moved between servers based on the matching.

公开(公告)号：US11946753B2

公开(公告)日：2024-04-02

申请号：US17364480

申请日：2021-06-30

Applicant: Adobe Inc.

Inventor： Fan Du ,  Sana Malik Lee ,  Georgios Theocharous ,  Eunyee Koh

IPC: H04W4/024 ,  G01C21/34 ,  G06Q10/047 ,  H04W4/021

CPC classification number: G01C21/343 ,  G01C21/3476 ,  G01C21/3484 ,  G06Q10/047 ,  H04W4/021 ,  H04W4/024

Abstract: The present disclosure relates to generating and modifying recommended event sequences utilizing a dynamic user preference interface. For example, in one or more embodiments, the system generates a recommended event sequence using a recommendation model trained based on a plurality of historical event sequences. The system then provides, for display via a client device, the recommendation, a plurality of interactive elements for entry of user preferences, and a visual representation of historical event sequences. Upon detecting input of user preferences, the system can modify a reward function of the recommendation model and provide a modified recommended event sequence together with the plurality of interactive elements. In one or more embodiments, as a user enters user preferences, the system additionally modifies the visual representation to display subsets of the plurality of historical event sequences corresponding to the preferences.

公开(公告)号：US20230394332A1

公开(公告)日：2023-12-07

申请号：US17804991

申请日：2022-06-01

Applicant: Adobe Inc.

Inventor： Jaron J.R. Lee ,  David Arbour ,  Georgios Theocharous

IPC: G06N5/04 ,  G06F11/34 ,  G06F11/30

CPC classification number: G06N5/04 ,  G06F11/3409 ,  G06F11/3006 ,  G06F11/3476

Abstract: The present disclosure describes methods, systems, and non-transitory computer-readable media for generating a projected value metric that projects a performance of a target policy within a digital action space. For instance, in one or more embodiments, the disclosed systems identify a target policy for performing digital actions represented within a digital action space. The disclosed systems further determine a set of sampled digital actions performed according to a logging policy and represented within the digital action space. Utilizing an embedding model, the disclosed systems generate a set of action embedding vectors representing the set of sampled digital actions within an embedding space. Further, utilizing the set of action embedding vectors, the disclosed systems generate a projected value metric indicating a projected performance of the target policy.

公开(公告)号：US11615293B2

公开(公告)日：2023-03-28

申请号：US16578863

申请日：2019-09-23

Applicant: ADOBE INC.

Inventor： Georgios Theocharous ,  Yash Chandak

IPC: G06N3/04 ,  G06N3/08

Abstract: Systems and methods are described for a decision-making process including actions characterized by stochastic availability, provide an Markov decision process (MDP) model that includes a stochastic action set based on the decision-making process, compute a policy function for the MDP model using a policy gradient based at least in part on a function representing the stochasticity of the stochastic action set, identify a probability distribution for one or more actions available at a time period using the policy function, and select an action for the time period based on the probability distribution.

公开(公告)号：US11501207B2

公开(公告)日：2022-11-15

申请号：US16578913

申请日：2019-09-23

Applicant: ADOBE INC.

Inventor： Georgios Theocharous ,  Yash Chandak

IPC: G06N20/00 ,  G06N7/00 ,  G06N5/04

Abstract: Systems and methods are described for a decision-making process that includes an increasing set of actions, compute a policy function for a Markov decision process (MDP) for the decision-making process, wherein the policy function is computed based on a state conditional function mapping states into an embedding space, an inverse dynamics function mapping state transitions into the embedding space, and an action selection function mapping the elements of the embedding space to actions, identify an additional set of actions in the increasing set of actions, update the inverse dynamics function based at least in part on the additional set of actions, update the policy function based on the updated inverse dynamics function and parameters learned during the computing the policy function, and select an action based on the updated policy function.

公开(公告)号：US20220261683A1

公开(公告)日：2022-08-18

申请号：US17174944

申请日：2021-02-12

Applicant: ADOBE INC.

Inventor： Tong Mu ,  Georgios Theocharous ,  David Arbour

IPC: G06N20/00 ,  G06N5/04 ,  G06N7/00

Abstract: Systems and methods for sequential recommendation are described. Embodiments receive a user interaction history including interactions of a user with a plurality of items, select a constraint from a plurality of candidate constraints based on lifetime values observed for the candidate constraints, wherein the lifetime values are based on items predicted for other users using a recommendation network subject to the candidate constraints, and predict a next item for the user based on the user interaction history using the recommendation network subject to the selected constraint.

公开(公告)号：US20220019888A1

公开(公告)日：2022-01-20

申请号：US16933361

申请日：2020-07-20

Applicant: Adobe Inc.

Inventor： Karan Aggarwal ,  Georgios Theocharous ,  Anup Rao

IPC: G06N3/08 ,  G06F9/54

Abstract: A single unified machine learning model (e.g., a neural network) is trained to perform both supervised event predictions and unsupervised time-varying clustering for a sequence of events (e.g., a sequence representing a user behavior) using sequences of events for multiple users using a combined loss function. The unified model can then be used for, given a sequence of events as input, predict a next event to occur after the last event in the sequence and generate a clustering result by performing a clustering operation on the sequence of events. As part of predicting the next event, the unified model is trained to predict an event type for the next event and a time of occurrence for the next event. In certain embodiments, the unified model is a neural network comprising a recurrent neural network (RNN) such as an Long Short Term Memory (LSTM) network.

公开(公告)号：US11062225B2

公开(公告)日：2021-07-13

申请号：US15373849

申请日：2016-12-09

Applicant: Adobe Inc.

Inventor： Nikolaos Vlassis ,  Georgios Theocharous

IPC: G06N7/00 ,  G06N20/00 ,  G06F16/9535 ,  G06N5/04

Abstract: Certain embodiments involve generating personalized recommendations for users by inferring a propensity of each individual user to accept a recommendation. For example, a system generates a personalized user model based on a historical transition matrix that provides state transition probabilities from a general population of users. The probabilities are adjusted based on the propensity for a user to accept a recommendation. The system determines a recommended action for the user to transition between predefined states based on the user model. Once the user has performed an activity that transitions from a current state, the system adjusts a probability distribution for an estimate of the propensity based on whether the activity is the recommended action.

公开(公告)号：US20210089868A1

公开(公告)日：2021-03-25

申请号：US16578863

申请日：2019-09-23

Applicant: ADOBE INC.

Inventor： Georgios Theocharous ,  Yash Chandak

IPC: G06N3/04 ,  G06N3/08

Abstract: Systems and methods are described for a decision-making process including actions characterized by stochastic availability, provide an Markov decision process (MDP) model that includes a stochastic action set based on the decision-making process, compute a policy function for the MDP model using a policy gradient based at least in part on a function representing the stochasticity of the stochastic action set, identify a probability distribution for one or more actions available at a time period using the policy function, and select an action for the time period based on the probability distribution.