Artificial Intelligence
The Artificial Intelligence (AI) Team is focused on research about computer vision to analyze human interaction through body movement and motion tracking for multiple persons in a 3-dimensional space.
Team Members
Team Lead
Aditya Puranik
Vrushabh Deogirikar
Julianna Dietrich
Ryan Fell
Shanice Liu
Eyitayo Makinde
James Vong
Utilizing the cutting-edge deep learning model to estimate the 3D pose of a person from any video dataset. A robust model is created through the merging and augmentation of many different datasets. Additionally, the project introduces many conversions where a 3D model is estimated for every 2D frame resulting in a more efficient 3D pose estimation.
Emotion detection is a critical challenge in intelligent human-robot interaction (HRI). We present two key contributions:
- MoEmo (Motion to Emotion), a cross-attention vision transformer (ViT) designed for detecting human emotions in robotics systems by analyzing 3D human pose estimations across diverse contexts.
- A comprehensive dataset featuring full-body videos of human movements, annotated with emotion labels derived from gestures and environmental contexts.
Using our Naturalistic Motion Database, we train MoEmo to jointly interpret motion and context, achieving emotion detection performance that surpasses the current state-of-the-art.