Skip to Main content Skip to Navigation
Journal articles

Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration

Abstract : The human–machine collaboration system is a key means of manufacturing. Its surveillance, prognostic, and health management are related to safety and manufacturing persistence. This paper begins with the mission requirements of intelligent manufacturing. The study is based on the visual question answering (VQA) technology with a digital twin to increase efficiency. The research contents are as follows: (1) A method of modeling human–machine collaboration based on digital twins is proposed. (2) A VQA is adopted in the digital twin. The video and neural language are considered. (3) VQA technology is introduced into the modeling of the human–machine collaboration system for consistent integration. With VQA technology, humans and machines can collaborate. Human–machine interaction and product counting are implemented in a case study to provide a comprehensive perception.
Document type :
Journal articles
Complete list of metadata

https://hal-utt.archives-ouvertes.fr/hal-03320661
Contributor : Jean-Baptiste Vu Van Connect in order to contact the contributor
Submitted on : Monday, August 16, 2021 - 11:39:47 AM
Last modification on : Wednesday, October 13, 2021 - 7:16:03 PM

Identifiers

Collections

UTT

`

Citation

Tian Wang, Jiakun Li, Zhaoning Kong, Xin Liu, Hichem Snoussi, et al.. Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration. Journal of Manufacturing Systems, Elsevier, 2021, 58 (PART B), pp.261-269. ⟨10.1016/j.jmsy.2020.07.011⟩. ⟨hal-03320661⟩

Share

Metrics

Record views

18