公开(公告)号：US20140369596A1

公开(公告)日：2014-12-18

申请号：US14099300

申请日：2013-12-06

Applicant: Purdue Research Foundation

Inventor： Jeffrey Mark Siskind ,  Andrei Barbu ,  Siddharth Narayanaswamy ,  Haonan Yu

IPC: G06K9/66

CPC classification number: G06K9/723 ,  G06F17/3079 ,  G06F17/30793 ,  G06K9/00751 ,  G06K9/00771 ,  G06K9/6297

Abstract: A method of testing a video against an aggregate query includes automatically receiving an aggregate query defining participant(s) and condition(s) on the participant(s). Candidate object(s) are detected in the frames of the video. A first lattice is constructed for each participant, the first-lattice nodes corresponding to the candidate object(s). A second lattice is constructed for each condition. An aggregate lattice is constructed using the respective first lattice(s) and the respective second lattice(s). Each aggregate-lattice node includes a scoring factor combining a first-lattice node factor and a second-lattice node factor. respective aggregate score(s) are determined of one or more path(s) through the aggregate lattice, each path including a respective plurality of the nodes in the aggregate lattice, to determine whether the video corresponds to the aggregate query. A method of providing a description of a video is also described and includes generating a candidate description with participant(s) and condition(s) selected from a linguistic model; constructing component lattices for the participant(s) or condition(s), producing an aggregate lattice having nodes combining component-lattice factors, and determining a score for the video with respect to the candidate description by determining an aggregate score for a path through the aggregate lattice. If the aggregate score does not satisfy a termination condition, participant(s) or condition(s) from the linguistic model are added to the condition, and the process is repeated. A method of testing a video against an aggregate query by mathematically optimizing a unified cost function is also described.

Abstract translation: 针对聚合查询测试视频的方法包括自动接收定义参与者的聚合查询和参与者的条件。 候选对象在视频的帧中被检测到。 为每个参与者构建第一格子，对应于候选对象的第一格子节点。 为每个条件构建第二个格子。 使用相应的第一晶格和相应的第二晶格构造聚集晶格。 每个聚集网格节点包括组合第一格格节点因子和第二格点节点因子的评分因子。 通过聚合格子确定一个或多个路径的各个聚合分数，每个路径包括聚合格子中的相应多个节点，以确定视频是否对应于聚合查询。 还描述了提供视频描述的方法，并且包括：从参与者和从语言模型中选择的条件生成候选描述; 为参与者或条件构建组件网格，产生具有组合分量 - 晶格因子的节点的聚集网格，以及通过确定通过所述候选描述的路径的总分数来确定所述视频相对于所述候选描述的得分 聚集格子。 如果总分不满足终止条件，则将来自语言模型的参与者或条件添加到条件中，并重复该过程。 还描述了通过数学优化统一成本函数来测试针对聚合查询的视频的方法。

公开(公告)号：US09183466B2

公开(公告)日：2015-11-10

申请号：US14099300

申请日：2013-12-06

Applicant: Purdue Research Foundation

Inventor： Jeffrey Mark Siskind ,  Andrei Barbu ,  Siddharth Narayanaswamy ,  Haonan Yu

IPC: G06K9/62 ,  G06K9/72 ,  G06K9/00 ,  G06F17/30

CPC classification number: G06K9/723 ,  G06F17/3079 ,  G06F17/30793 ,  G06K9/00751 ,  G06K9/00771 ,  G06K9/6297

Abstract: A method of testing a video against an aggregate query includes automatically receiving an aggregate query defining participant(s) and condition(s) on the participant(s). Candidate object(s) are detected in the frames of the video. A first lattice is constructed for each participant, the first-lattice nodes corresponding to the candidate object(s). A second lattice is constructed for each condition. An aggregate lattice is constructed using the respective first lattice(s) and the respective second lattice(s). Each aggregate-lattice node includes a scoring factor combining a first-lattice node factor and a second-lattice node factor. respective aggregate score(s) are determined of one or more path(s) through the aggregate lattice, each path including a respective plurality of the nodes in the aggregate lattice, to determine whether the video corresponds to the aggregate query. A method of providing a description of a video is also described and includes generating a candidate description with participant(s) and condition(s) selected from a linguistic model; constructing component lattices for the participant(s) or condition(s), producing an aggregate lattice having nodes combining component-lattice factors, and determining a score for the video with respect to the candidate description by determining an aggregate score for a path through the aggregate lattice. If the aggregate score does not satisfy a termination condition, participant(s) or condition(s) from the linguistic model are added to the condition, and the process is repeated. A method of testing a video against an aggregate query by mathematically optimizing a unified cost function is also described.

Abstract translation: 针对聚合查询测试视频的方法包括自动接收定义参与者的聚合查询和参与者的条件。 候选对象在视频的帧中被检测到。 为每个参与者构建第一格子，对应于候选对象的第一格子节点。 为每个条件构建第二个格子。 使用相应的第一晶格和相应的第二晶格构造聚集晶格。 每个聚集网格节点包括组合第一格格节点因子和第二格点节点因子的评分因子。 通过聚合格子确定一个或多个路径的各个聚合分数，每个路径包括聚合格子中的相应多个节点，以确定视频是否对应于聚合查询。 还描述了提供视频描述的方法，并且包括：从参与者和从语言模型中选择的条件生成候选描述; 为参与者或条件构建组件网格，产生具有组合分量 - 晶格因子的节点的聚集网格，以及通过确定通过所述候选描述的路径的总分数来确定所述视频相对于所述候选描述的得分 聚集格子。 如果总分不满足终止条件，则将来自语言模型的参与者或条件添加到条件中，并重复该过程。 还描述了通过数学优化统一成本函数来测试针对聚合查询的视频的方法。