Invention Publication
- Patent Title: METHOD OF IMAGE RECONSTRUCTION FOR CROSS-MODAL COMMUNICATION SYSTEM AND DEVICE THEREOF
-
Application No.: US18002500Application Date: 2022-07-01
-
Publication No.: US20230196633A1Publication Date: 2023-06-22
- Inventor: Liang ZHOU , Xin Wei , Meng Zhang , Jianxin Chen , Zihan Gao
- Applicant: NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS
- Applicant Address: CN Nanjing
- Assignee: NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS
- Current Assignee: NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS
- Current Assignee Address: CN Nanjing
- Priority: CN 2110776726.4 2021.07.09
- International Application: PCT/CN2022/103303 2022.07.01
- Date entered country: 2022-12-20
- Main IPC: G06T11/00
- IPC: G06T11/00 ; G08B6/00 ; G06N3/0464 ; G06N3/045 ; G06N3/048 ; G06N3/094

Abstract:
A method of image reconstruction for a cross-modal communication system is disclosed. The method reconstructs a damaged, lost, or delayed image signal during transmission by using complete haptic signals received by a receiving end in the cross-modal communication system, and further constructs a cross-modal interaction network with reference to an attention mechanism, thus solving the limitation of the conventional generation model that it can only be trained on paired samples. An image reconstruction device for a cross-modal communication system is also disclosed. By fully utilizing semantic correlation between different-modality data and realizing cross-modal generation from haptic signals to image signals for unpaired data, the present invention overcomes the difficulty in acquiring haptic-image signal pairs in the practical cross-modal communication system, and significantly improves the quality and class accuracy of generated image signals.
Public/Granted literature
- US11748919B2 Method of image reconstruction for cross-modal communication system and device thereof Public/Granted day:2023-09-05
Information query
IPC分类:
G | 物理 |
G06 | 计算;推算或计数 |
G06T | 一般的图像数据处理或产生 |
G06T11/00 | 2D〔二维〕图像的生成 |