Pose-Guided Inflated 3D ConvNet for action recognition in videos - Archive ouverte HAL Access content directly
Journal Articles Signal Processing: Image Communication Year : 2021

Pose-Guided Inflated 3D ConvNet for action recognition in videos

(1) , (1, 2) , (2) , (3) , (1) , (1) , (4)
1
2
3
4

Abstract

Human action recognition in videos is still an important while challenging task. Existing methods based on RGB image or optical flow are easily affected by clutters and ambiguous backgrounds. In this paper, we propose a novel Pose-Guided Inflated 3D ConvNet framework (PI3D) to address this issue. First, we design a spatial–temporal pose module, which provides essential clues for the Inflated 3D ConvNet (I3D). The pose module consists of pose estimation and pose-based action recognition. Second, for multi-person estimation task, the introduced pose estimation network can determine the action most relevant to the action category. Third, we propose a hierarchical pose-based network to learn the spatial–temporal features of human pose. Moreover, the pose-based network and I3D network are fused at the last convolutional layer without loss of performance. Finally, the experimental results on four data sets (HMDB-51, SYSU 3D, JHMDB and Sub-JHMDB) demonstrate that the proposed PI3D framework outperforms the existing methods on human action recognition. This work also shows that posture cues significantly improve the performance of I3D.
Not file

Dates and versions

hal-03320743 , version 1 (16-08-2021)

Identifiers

Cite

Qianyu Wu, Aichun Zhu, Ran Cui, Tian Wang, Fangqiang Hu, et al.. Pose-Guided Inflated 3D ConvNet for action recognition in videos. Signal Processing: Image Communication, 2021, 91, pp.116098. ⟨10.1016/j.image.2020.116098⟩. ⟨hal-03320743⟩
42 View
0 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More