The Artificial Intelligence (AI) Team is focused on research about computer vision to analyze human interaction through body movement and motion tracking for multiple persons in a 3-dimensional space.
Utilizing the cutting-edge deep learning model to estimate the 3d pose of a person from any video dataset. A robust model is created through the merging of many different datasets and augmentation of the datasets. Additionally, the project introduces many to many conversions where a 3d model is estimated for every 2d frame resulting in a more efficient 3d pose estimation.
Emotion detection presents challenges to intelligent human-robot interaction (HRI).
We introduce two things:
1) MoEmo (Motion to Emotion), a cross-attention vision transformer (ViT) for human emotion detection within robotics systems based on 3D human pose estimations across various contexts, and
2) a data set that offers full-body videos of human movement and corresponding emotion labels based on human gestures and environmental contexts.
Leveraging our Naturalistic Motion Database, we train the MoEmo system to jointly analyze motion and context, yielding emotion detection that outperforms the current state-of-the-art.