Santa Clara University

SCU Engineering News Banner

The Language of Learning

When Maria Pantoja, lecturer in the Department of Computer Engineering, was the featured speaker last year at a weekly spring quarter faculty forum on teaching innovation, a fruitful collaboration and a new teaching tool were born. Pantoja's presentation focused on her use of a video game application to overcome students' fears about learning computer programming. In the audience that day was Marie Bertola, lecturer of Italian in the Department of Modern Languages and Literatures, who recalls, "I had this vague idea for using a computer app to help students acquire and practice speaking skills in a way that is convenient and user-friendly, so I was looking for someone with the technological know-how to help implement it." She found a perfect partner in Pantoja.

Lecturers Marie Bertola and Maria Pantoja
Lecturers Marie Bertola and Maria Pantoja
Photo: Charles Barry, University Photographer

As neither are native English speakers (Bertola was born in Italy, while Pantoja hails from Spain), both professors have first-hand experience with the challenges of learning a new language and perfecting its pronunciation. "Technology is impacting the teaching of language at different levels today," said Bertola. "We use e-workbooks so students of language can get effective feedback on written exercises, but so far the computer has not been able to give immediate and reliable feedback when students speak."

Supported by an internal research grant from SCU, Bertola enlisted volunteer students' help to video a wide range of speakers. While the recordings displayed completely different movements, some consistencies were found that informed Pantoja's work to refine the program. "It took all summer to determine which facial movements were the best indicators of proper pronunciation. Mustaches can hide the lips, not everyone has the same crease along the side of the mouth, and forget about noses," quipped Pantoja. "In the end, we decided to concentrate on four points of the mouth that are statistically very consistent and can provide a precise tool for improving pronunciation."

The end product will be an interface that is visual, auditory, and tactile. The program will analyze the speaker's voice and facial movement, compare them with those of a standard speaker, and then provide recommendations for improvement. "In 'olden times,' I would ask students to get a pocket mirror to help internalize the connection between how the mouth looks and how the words sound—much like the way a ballerina uses a mirror to properly position and control every muscle. This app will give feedback that guides those speaking and lets them know if they've got it right or if they should try again using specific modifications—all in a nonthreatening, nonjudgmental manner," said Bertola.

Pantoja is excited about the pedagogical potential for the app—not just from the students' perspective but for professors, as well. "The algorithm will keep learning, and the databank we are creating will provide us with feedback on our own work of teaching, giving us information on what students struggle with and how best to use our time in class," she said. With an audio version ready to demo now, she and Bertola will continue to develop the app. "We will never finish; I can see a new upgrade every year," said Pantoja, who continues, "Collaboration with the modern language department is not one that has traditionally been done. Hopefully, this is just the tip of the iceberg of what we can accomplish together."