公开(公告)号：US20210192972A1

公开(公告)日：2021-06-24

申请号：US17129541

申请日：2020-12-21

Applicant: SRI International

Inventor： Girish Acharya ,  Louise Yarnall ,  Anirban Roy ,  Michael Wessel ,  Yi Yao ,  John J. Byrnes ,  Dayne Freitag ,  Zachary Weiler ,  Paul Kalmar

IPC: G09B19/00 ,  G06K9/72 ,  G10L15/18 ,  G06K9/00 ,  G10L25/57 ,  G06K9/62 ,  G09B5/06 ,  G06N20/00

Abstract: This disclosure describes machine learning techniques for capturing human knowledge for performing a task. In one example, a video device obtains video data of a first user performing the task and one or more sensors generate sensor data during performance of the task. An audio device obtains audio data describing performance of the task. A computation engine applies a machine learning system to correlate the video data to the audio data and sensor data to identify portions of the video, sensor, and audio data that depict a same step of a plurality of steps for performing the task. The machine learning system further processes the correlated data to update a domain model defining performance of the task. A training unit applies the domain model to generate training information for performing the task. An output device outputs the training information for use in training a second user to perform the task.

公开(公告)号：US12118773B2

公开(公告)日：2024-10-15

申请号：US17129541

申请日：2020-12-21

Applicant: SRI International

Inventor： Girish Acharya ,  Louise Yarnall ,  Anirban Roy ,  Michael Wessel ,  Yi Yao ,  John J. Byrnes ,  Dayne Freitag ,  Zachary Weiler ,  Paul Kalmar

IPC: G06V10/82 ,  G06F18/22 ,  G06N20/00 ,  G06V20/20 ,  G06V20/40 ,  G06V30/19 ,  G06V30/262 ,  G06V40/10 ,  G06V40/20 ,  G09B5/06 ,  G09B19/00 ,  G10L15/18 ,  G10L25/57

CPC classification number: G06V10/82 ,  G06F18/22 ,  G06N20/00 ,  G06V20/20 ,  G06V20/41 ,  G06V30/19173 ,  G06V30/274 ,  G06V40/10 ,  G06V40/113 ,  G06V40/28 ,  G09B5/065 ,  G09B19/003 ,  G10L15/1815 ,  G10L25/57

Abstract: This disclosure describes machine learning techniques for capturing human knowledge for performing a task. In one example, a video device obtains video data of a first user performing the task and one or more sensors generate sensor data during performance of the task. An audio device obtains audio data describing performance of the task. A computation engine applies a machine learning system to correlate the video data to the audio data and sensor data to identify portions of the video, sensor, and audio data that depict a same step of a plurality of steps for performing the task. The machine learning system further processes the correlated data to update a domain model defining performance of the task. A training unit applies the domain model to generate training information for performing the task. An output device outputs the training information for use in training a second user to perform the task.