公开(公告)号：US20240355140A1

公开(公告)日：2024-10-24

申请号：US18038611

申请日：2022-06-20

Applicant: DALIAN UNIVERSITY OF TECHNOLOGY

Inventor： Xin YANG ,  Xiaopeng WEI ,  Bo DONG ,  Haiwei ZHANG

IPC: G06V40/20 ,  G06T7/20 ,  G06T7/70 ,  G06V10/764 ,  G06V10/77 ,  G06V10/82 ,  G06V40/18

CPC classification number: G06V40/20 ,  G06T7/20 ,  G06T7/70 ,  G06V10/764 ,  G06V10/7715 ,  G06V10/82 ,  G06V40/193 ,  G06T2207/30201

Abstract: The present invention belongs to the technical field of computer vision, and proposes a lightweight real-time emotion analysis method incorporating eye tracking. In the method, gray frames and event frames that have synchronized time are acquired through event-based cameras and respectively input to a frame branch and an event branch; the frame branch extracts spatial features by convolution operations, and the event branch extracts temporal features through conv-SNN blocks; the frame branch has a guide attention mechanism for the event branch; and the spatial features and the temporal features are integrated by fully connected layers, The final output is the average of the n fully connected layer outputs, which represents the final expression. The method can recognize the emotional expression of any stage in various complex light changing scenarios; and in the case of limited accuracy loss, the emotion recognition time is shortened to achieve “real-time” user emotion analysis.