公开(公告)号：US20160110433A1

公开(公告)日：2016-04-21

申请号：US14974871

申请日：2015-12-18

Applicant: SRI International

Inventor： Harpreet Singh Sawhney ,  Jayakrishnan Eledath ,  Ajay Divakaran ,  Mayank Bansal ,  Hui Cheng

IPC: G06F17/30

CPC classification number: G06F17/30867 ,  G06F17/2785 ,  G06F17/2827 ,  G06F17/3087 ,  G06Q50/26

Abstract: Methods and apparatuses of the present invention generally relate to generating actionable data based on multimodal data from unsynchronized data sources. In an exemplary embodiment, the method comprises receiving multimodal data from one or more unsynchronized data sources, extracting concepts from the multimodal data, the concepts comprising at least one of objects, actions, scenes and emotions, indexing the concepts for searchability; and generating actionable data based on the concepts.

Abstract translation: 本发明的方法和装置通常涉及基于来自不同步数据源的多模态数据生成可操作数据。 在示例性实施例中，该方法包括从一个或多个非同步数据源接收多模态数据，从多模态数据中提取概念，所述概念包括对象，动作，场景和情绪中的至少一个，为可搜索性索引概念; 并基于这些概念生成可操作的数据。

公开(公告)号：US09268994B2

公开(公告)日：2016-02-23

申请号：US13967521

申请日：2013-08-15

Applicant: SRI International

Inventor： Ishani Chakraborty ,  Hui Cheng ,  Omar Javed

IPC: G06K9/00

CPC classification number: G06K9/00248 ,  G06K9/00221 ,  G06K9/00677

Abstract: A unified framework detects and classifies people interactions in unconstrained user generated images. Previous approaches directly map people/face locations in two-dimensional image space into features for classification. Among other things, the disclosed framework estimates a camera viewpoint and people positions in 3D space and then extracts spatial configuration features from explicit three-dimensional people positions.

Abstract translation: 统一的框架可以检测和分类人们在无约束用户生成的图像中的交互。 以前的方法直接将二维图像空间中的人物/面部位置映射到要分类的特征中。 除此之外，所公开的框架估计了3D空间中的摄像机视点和人员位置，然后从显式的三维人员位置提取空间配置特征。

公开(公告)号：US20130282747A1

公开(公告)日：2013-10-24

申请号：US13737607

申请日：2013-01-09

Applicant: SRI INTERNATIONAL

Inventor： Hui Cheng ,  Harpreet Singh Sawhney ,  Ajay Divakaran ,  Qian Yu ,  Jingen Liu ,  Amir Tamrakar ,  Saad Ali ,  Omar Javed

IPC: G06F17/30

CPC classification number: G06F17/30823 ,  G06F17/30023 ,  G06F17/30784 ,  G06F17/30817

Abstract: A complex video event classification, search and retrieval system can generate a semantic representation of a video or of segments within the video, based on one or more complex events that are depicted in the video, without the need for manual tagging. The system can use the semantic representations to, among other things, provide enhanced video search and retrieval capabilities.

Abstract translation: 复杂的视频事件分类，搜索和检索系统可以基于视频中描绘的一个或多个复杂事件，而不需要手动标记来生成视频中的视频或片段的语义表示。 该系统可以使用语义表示来提供增强的视频搜索和检索功能。

公开(公告)号：US10963504B2

公开(公告)日：2021-03-30

申请号：US16077449

申请日：2017-02-13

Applicant: SRI International

Inventor： Hui Cheng ,  Jingen Liu ,  Harpreet Sawhney ,  Mohamed Elhoseiny

IPC: G06F16/783 ,  G06F16/435 ,  G06F40/30 ,  G10L15/26 ,  G06K9/34

Abstract: Zero-shot content detection includes building/training a semantic space by embedding word-based document descriptions of a plurality of documents into a multi-dimensional space using a semantic embedding technique; detecting a plurality of features in the multimodal content by applying feature detection algorithms to the multimodal content; determining respective word-based concept descriptions for concepts identified in the multimodal content using the detected features; embedding the respective word-based concept descriptions into the semantic space; and in response to a content detection action, (i) embedding/mapping words representative of the content detection action into the semantic space, (ii) automatically determining, without the use of training examples, concepts in the semantic space relevant to the content detection action based on the embedded words, and (iii) identifying portions of the multimodal content responsive to the content detection action based on the concepts in the semantic space determined to be relevant to the content detection action.

公开(公告)号：US20190065492A1

公开(公告)日：2019-02-28

申请号：US16077449

申请日：2017-02-13

Applicant: SRI International

Inventor： Hui Cheng ,  Jingen Liu ,  Harpreet Sawhney ,  Mohamed Elhoseiny

IPC: G06F17/30 ,  G06F17/27 ,  G06K9/34 ,  G10L15/26

Abstract: Zero-shot content detection includes building/training a semantic space by embedding word-based document descriptions of a plurality of documents into a multi-dimensional space using a semantic embedding technique; detecting a plurality of features in the multimodal content by applying feature detection algorithms to the multimodal content; determining respective word-based concept descriptions for concepts identified in the multimodal content using the detected features; embedding the respective word-based concept descriptions into the semantic space; and in response to a content detection action, (i) embedding/mapping words representative of the content detection action into the semantic space, (ii) automatically determining, without the use of training examples, concepts in the semantic space relevant to the content detection action based on the embedded words, and (iii) identifying portions of the multimodal content responsive to the content detection action based on the concepts in the semantic space determined to be relevant to the content detection action.

公开(公告)号：US10198509B2

公开(公告)日：2019-02-05

申请号：US15005795

申请日：2016-01-25

Applicant: SRI INTERNATIONAL

Inventor： Hui Cheng ,  Harpreet Singh Sawhney ,  Ajay Divakaran ,  Qian Yu ,  Jingen Liu ,  Amir Tamrakar ,  Saad Ali ,  Omar Javed

IPC: G06F17/30

Abstract: A complex video event classification, search and retrieval system can generate a semantic representation of a video or of segments within the video, based on one or more complex events that are depicted in the video, without the need for manual tagging. The system can use the semantic representations to, among other things, provide enhanced video search and retrieval capabilities.

公开(公告)号：US20160379062A1

公开(公告)日：2016-12-29

申请号：US14519927

申请日：2014-10-21

Applicant: SRI International

Inventor： Saad Masood Khan ,  Hui Cheng ,  Dennis Lee Matthies ,  Harpreet Singh Sawhney ,  Sang-Hack Jung ,  Chris Broaddus ,  Bogdan Calin Mihai Matei ,  Ajay Divakaran

IPC: G06K9/00 ,  G06T7/20 ,  B64D47/08 ,  G06K9/46 ,  B64C39/02 ,  G06T7/00 ,  G06K9/62

CPC classification number: G06K9/00785 ,  B64C39/024 ,  B64C2201/123 ,  B64D47/08 ,  G06K9/00651 ,  G06K9/4671 ,  G06K9/6269 ,  G06K9/6282 ,  G06T7/11 ,  G06T7/20 ,  G06T2207/10012

Abstract: A computer implemented method for determining a vehicle type of a vehicle detected in an image is disclosed. An image having a detected vehicle is received. A number of vehicle models having salient feature points is projected on the detected vehicle. A first set of features derived from each of the salient feature locations of the vehicle models is compared to a second set of features derived from corresponding salient feature locations of the detected vehicle to form a set of positive match scores (p-scores) and a set of negative match scores (n-scores). The detected vehicle is classified as one of the vehicle models models based at least in part on the set of p-scores and the set of n-scores.

Abstract translation: 公开了一种用于确定在图像中检测到的车辆的车辆类型的计算机实现的方法。 接收具有检测到的车辆的图像。 在检测到的车辆上投影出具有显着特征点的多个车辆模型。 将来自车辆模型的每个突出特征位置的第一组特征与从所检测到的车辆的相应突出特征位置导出的第二组特征进行比较，以形成一组正的匹配得分（p分数）和 一组负面比赛得分（n分）。 检测到的车辆至少部分地基于p分数集合和n分数集合被分类为车辆模型模型之一。

公开(公告)号：US09330296B2

公开(公告)日：2016-05-03

申请号：US14021696

申请日：2013-09-09

Applicant: SRI International

Inventor： Ishani Chakraborty ,  Hui Cheng ,  Omar Javed

IPC: G06K9/00

CPC classification number: G06K9/00677 ,  G06K9/00221 ,  G06K9/00664

Abstract: An entity interaction recognition system algorithmically recognizes a variety of different types of entity interactions that may be captured in two-dimensional images. In some embodiments, the system estimates the three-dimensional spatial configuration or arrangement of entities depicted in the image. In some embodiments, the system applies a proxemics-based analysis to determine an interaction type. In some embodiments, the system infers, from a characteristic of an entity detected in an image, an area or entity of interest in the image.

Abstract translation: 实体交互识别系统在算法上识别可以在二维图像中捕获的各种不同类型的实体交互。 在一些实施例中，系统估计在图像中描绘的实体的三维空间配置或排列。 在一些实施例中，系统应用基于proxemics的分析来确定交互类型。 在一些实施例中，系统从图像中检测到的实体的特征推断图像中感兴趣的区域或实体。

公开(公告)号：US20160004911A1

公开(公告)日：2016-01-07

申请号：US14846318

申请日：2015-09-04

Applicant: SRI International

Inventor： Hui Cheng ,  Ajay Divakaran ,  Elizabeth Shriberg ,  Harpreet Singh Sawhney ,  Jingen Liu ,  Ishani Chakraborty ,  Omar Javed ,  David Chisolm ,  Behjat Siddiquie ,  Steven S. Weiner

IPC: G06K9/00 ,  H04N21/44 ,  H04N21/8549 ,  H04N21/439 ,  H04N21/233 ,  H04N21/234

Abstract: A computing system for recognizing salient events depicted in a video utilizes learning algorithms to detect audio and visual features of the video. The computing system identifies one or more salient events depicted in the video based on the audio and visual features.

Abstract translation: 用于识别视频中描绘的突出事件的计算系统利用学习算法来检测视频的音频和视觉特征。 计算系统基于音频和视觉特征识别视频中描绘的一个或多个突出事件。

公开(公告)号：US20140270482A1

公开(公告)日：2014-09-18

申请号：US14021696

申请日：2013-09-09

Applicant: SRI International

Inventor： Ishani Chakraborty ,  Hui Cheng ,  Omar Javed

IPC: G06K9/00

CPC classification number: G06K9/00677 ,  G06K9/00221 ,  G06K9/00664

Abstract: An entity interaction recognition system algorithmically recognizes a variety of different types of entity interactions that may be captured in two-dimensional images. In some embodiments, the system estimates the three-dimensional spatial configuration or arrangement of entities depicted in the image. In some embodiments, the system applies a proxemics-based analysis to determine an interaction type. In some embodiments, the system infers, from a characteristic of an entity detected in an image, an area or entity of interest in the image.

Abstract translation: 实体交互识别系统在算法上识别可以在二维图像中捕获的各种不同类型的实体交互。 在一些实施例中，系统估计在图像中描绘的实体的三维空间配置或排列。 在一些实施例中，系统应用基于proxemics的分析来确定交互类型。 在一些实施例中，系统从图像中检测到的实体的特征推断图像中感兴趣的区域或实体。