Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration - Archive ouverte HAL Access content directly
Journal Articles Journal of Manufacturing Systems Year : 2021

Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration

(1) , (2) , , (3) , (4) , (5)
1
2
3
4
5

Abstract

The human–machine collaboration system is a key means of manufacturing. Its surveillance, prognostic, and health management are related to safety and manufacturing persistence. This paper begins with the mission requirements of intelligent manufacturing. The study is based on the visual question answering (VQA) technology with a digital twin to increase efficiency. The research contents are as follows: (1) A method of modeling human–machine collaboration based on digital twins is proposed. (2) A VQA is adopted in the digital twin. The video and neural language are considered. (3) VQA technology is introduced into the modeling of the human–machine collaboration system for consistent integration. With VQA technology, humans and machines can collaborate. Human–machine interaction and product counting are implemented in a case study to provide a comprehensive perception.
Not file

Dates and versions

hal-03320661 , version 1 (16-08-2021)

Identifiers

Cite

Tian Wang, Jiakun Li, Zhaoning Kong, Xin Liu, Hichem Snoussi, et al.. Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration. Journal of Manufacturing Systems, 2021, 58 (PART B), pp.261-269. ⟨10.1016/j.jmsy.2020.07.011⟩. ⟨hal-03320661⟩
23 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More