公开(公告)号：US11790213B2

公开(公告)日：2023-10-17

申请号：US16439508

申请日：2019-06-12

Applicant: SRI International

Inventor： Yi Yao ,  Ajay Divakaran ,  Pallabi Ghosh

IPC: G06N3/045 ,  G06N3/08

CPC classification number: G06N3/045 ,  G06N3/08

Abstract: Techniques are disclosed for identifying multimodal subevents within an event having spatially-related and temporally-related features. In one example, a system receives a Spatio-Temporal Graph (STG) comprising (1) a plurality of nodes, each node having a feature descriptor that describes a feature present in the event, (2) a plurality of spatial edges, each spatial edge describing a spatial relationship between two of the plurality of nodes, and (3) a plurality of temporal edges, each temporal edge describing a temporal relationship between two of the plurality of nodes. Furthermore, the STG comprises at least one of: (1) variable-length descriptors for the feature descriptors or (2) temporal edges that span multiple time steps for the event. A machine learning system processes the STG to identify the multimodal subevents for the event. In some examples, the machine learning system comprises stacked Spatio-Temporal Graph Convolutional Networks (STGCNs), each comprising a plurality of STGCN layers.