Skip to Main content Skip to Navigation
Journal articles

Pose-Guided Inflated 3D ConvNet for action recognition in videos

Abstract : Human action recognition in videos is still an important while challenging task. Existing methods based on RGB image or optical flow are easily affected by clutters and ambiguous backgrounds. In this paper, we propose a novel Pose-Guided Inflated 3D ConvNet framework (PI3D) to address this issue. First, we design a spatial–temporal pose module, which provides essential clues for the Inflated 3D ConvNet (I3D). The pose module consists of pose estimation and pose-based action recognition. Second, for multi-person estimation task, the introduced pose estimation network can determine the action most relevant to the action category. Third, we propose a hierarchical pose-based network to learn the spatial–temporal features of human pose. Moreover, the pose-based network and I3D network are fused at the last convolutional layer without loss of performance. Finally, the experimental results on four data sets (HMDB-51, SYSU 3D, JHMDB and Sub-JHMDB) demonstrate that the proposed PI3D framework outperforms the existing methods on human action recognition. This work also shows that posture cues significantly improve the performance of I3D.
Document type :
Journal articles
Complete list of metadata

https://hal-utt.archives-ouvertes.fr/hal-03320743
Contributor : Jean-Baptiste Vu Van Connect in order to contact the contributor
Submitted on : Monday, August 16, 2021 - 1:28:13 PM
Last modification on : Wednesday, October 13, 2021 - 7:16:03 PM

Identifiers

Collections

UTT

`

Citation

Qianyu Wu, Aichun Zhu, Ran Cui, Tian Wang, Fangqiang Hu, et al.. Pose-Guided Inflated 3D ConvNet for action recognition in videos. Signal Processing: Image Communication, Elsevier, 2021, 91, pp.116098. ⟨10.1016/j.image.2020.116098⟩. ⟨hal-03320743⟩

Share

Metrics

Record views

17