Skip to main content

Artificial Intelligence

Artificial Intelligence

The Artificial Intelligence (AI) Team is focused on research about computer vision to analyze human interaction through body movement and motion tracking for multiple persons in a 3-dimensional space. 

Team Members

Team Lead

Aditya Puranik

Vrushabh Deogirikar

Julianna Dietrich

Ryan Fell

Shanice Liu

Eyitayo Makinde

James Vong

Utilizing the cutting-edge deep learning model to estimate the 3D pose of a person from any video dataset. A robust model is created through the merging and augmentation of many different datasets. Additionally, the project introduces many conversions where a 3D model is estimated for every 2D frame resulting in a more efficient 3D pose estimation.

Emotion detection is a critical challenge in intelligent human-robot interaction (HRI). We present two key contributions:

  1. MoEmo (Motion to Emotion), a cross-attention vision transformer (ViT) designed for detecting human emotions in robotics systems by analyzing 3D human pose estimations across diverse contexts.
  2. A comprehensive dataset featuring full-body videos of human movements, annotated with emotion labels derived from gestures and environmental contexts.

Using our Naturalistic Motion Database, we train MoEmo to jointly interpret motion and context, achieving emotion detection performance that surpasses the current state-of-the-art.

MoEmo Vision Transformer: Integrating Cross-Attention and Movement Vectors in 3D Pose Estimation for HRI Emotion Detection