Santa Clara University

Course Evaluation Pilot 

• Compare and Contrast
• Overview of SALG: Student Assessment of Learning Gains
• Overview of IDEA: Individual Development & Educational Assessment
• Sample SALG for Cynthia Baker's SCTR 19 course
• Sample SALG for Jane Curry's POLI 2 course

 

Stephen Carroll, a lecturer in the English Department with several years experience designing assessment instruments is working with a task force appointed by the University Coordinating Committee, an informal group of faculty, to investigate possible alternatives to SCU's course evaluation system.

Overview

Santa Clara University currently uses two different forms to evaluate faculty and their courses. The first form, used by the College of Arts and Sciences and the College of Engineering was developed in-house in the early 1970s. The Business School revised the form in the mid 1990s, and now uses the updated version. In the years the forms have been in use, faculty members have posed many questions about the forms and their use. Do the forms ask the right questions? Do the answers they gather help us improve our courses? Do they help us better understand what is happening in our classrooms? Do they help us evaluate and improve our teaching? Do they help us improve student learning?   

Dolores De LaGuardia

                                                

In response to these questions and others, then-Faculty Senate President Simone Billings convened an ad hoc group of faculty members in Winter Quarter 2004 to study the current system for conducting course evaluations and to consider possible alternatives. After considerable discussion of the issues, the working group decided to survey course evaluation practices at a number of benchmark institutions and to attempt to develop a best-practices model. They appointed Stephen Carroll to assist them with their research.

Dr. Carroll studied course evaluation systems at eight benchmark institutions, examined the American Association of University Professors’ recommendations on faculty evaluation, and surveyed eight of the leading commercial assessment instruments. He delivered his report to the working group in March. The group discussed the report at a couple of meetings, and decided to conduct a small pilot program to compare SCU’s current course evaluation system with two top-rated systems used on campuses around the United States and Canada. Those systems were the IDEA (Individual Development and Educational Assessment) system, developed at Kansas State University, and the SALG (Student Assessment of Learning Gains), created by SENCER (Scientists Engaged in New Civic and Environmental Responsibilities), a project developed at the American Association of Colleges and Universities with grant support by the National Science Foundation.

The pilot was successful: participating faculty reported that the new systems provided them with more detailed and more useful information about their teaching than the older system. They were pleased that the new instruments focused on student learning rather than on students’ perceptions of the teacher’s behavior. This focus on student learning substantially mitigates the well-known problem that students tend to rate female and minority instructors lower than white males on surveys that ask about teacher’s behaviors. Many faculty also commented that they appreciated the flexibility offered by the new instruments—they liked being able to adjust the survey to reflect their own teaching goals and methods.

Because they were using these systems for the first time, however, some participants said they didn’t fully understand what the results they received meant. They wanted to repeat the pilot so that they could improve their understanding of the data. The participants and the working group agreed that the pilot should be repeated and that it should be expanded to include more faculty members. They wanted to broaden the base of participants to validate their conclusions and to help them decide how best to proceed. Perhaps even more important, they wanted to share the greater insights provided by the new instruments.

You are therefore invited to participate in the continuing pilot. This site provides an overview of the two instruments—the IDEA and the SALG, explains the relative advantages and disadvantages of each, and furnishes instructions on how to use them. There are also links to the IDEA and SALG websites if you want further information. Please feel free to look around. Explore, think, experiment. If you have questions, please send them to Stephen Carroll (scarroll@scu.edu). If you decide you would like to participate in the pilot, you are welcome to sign up online. You may also sign up by contacting Stephen Carroll in the English Department.