Overview of the IDEA (Individual Development & Educational Assessment) system:
The IDEA system is robust and well-developed. It originated in 1968-9 at Kansas State University , and is now used on over 200 campuses in over forty states and Canada . The instrument is “based on in-depth research and leverages best practices in the area.”2 It is pedagogically, technically, and statistically very sophisticated and there is an extensive body of research explaining and supporting it.
It consists of two basic forms: a Faculty Information Form (FIF), and a student Diagnostic Form. (There is a shorter version of the diagnostic form called--surprisingly enough—the Short Form, which may be used in place of the Diagnostic Form. It elides most of the questions designed to support teaching and learning improvement, and IDEA recommends that it only be used to support personnel decisions.)
At the beginning of the quarter, each faculty member fills out a Faculty Information Form for each course she or he is teaching. On the FIF they specify a small number of learning goals as essential or important for this specific course. Faculty are also asked to assess various contextual factors which IDEA researchers have found to affect scores independently of faculty behavior. At the end of the term, students fill out Diagnostic Forms. The Diagnostic Form asks twenty questions about the frequency with which the instructor used various teaching methods and twelve questions in which students rate their progress on learning objectives. Three questions compare the present course to other classes they have taken in regard to amount of reading required, amount of other work required, and difficulty of subject matter. Seven questions survey students’ attitudes about the course and the instructor, and seven questions relate to the students’ and instructor’s overall approach to teaching and learning.
The raw data collected on the Diagnostic Forms is filtered through the data collected by the FIF to give statistics about the effectiveness of the faculty member in accomplishing the goals they set for themselves. The contextual factors are used to improve fairness by removing from consideration factors which are beyond the faculty member’s control.
The IDEA system is available in both online and paper-based versions.
Advantages of the IDEA system:
- Because the faculty member chooses from a list of twelve learning objectives on the FIF, the IDEA system is potentially less time-consuming than the SALG early in the term.
- The IDEA system offers very sophisticated and detailed reports, providing data with a documented, high degree of validity and reliability.
- The reports include specific pedagogical suggestions in areas where the faculty member’s performance is below average.
- Their extremely large database makes possible summary reports for groups, national norming of scores, and comprehensive longitudinal reporting.
- The IDEA system has a well-developed system for controlling for outside influences on scores, making the ratings it provides fairer than most.
Disadvantages of the IDEA system:
- The system has only limited flexibility. Although one may add to the existing 47 questions on the Diagnostic Form, one may not delete or change any of the standard questions.
- Some students and faculty at SCU have mentioned that they are bothered by the fact that some of the standard questions do not apply to their course and there is no way to indicate that on the form.
- Some faculty at SCU and other institutions have reported that they find the reports they receive at the end of the term difficult to understand and use.
- Many faculty at SCU and other institutions have expressed concern about how the information provided by the report will be used by chairs and administrators. Because the IDEA system produces numerical results, many faculty worry that those numbers will be used in ways not justified by their derivation, validity and reliability. The IDEA Center ’s own research on this subject suggests that this is a not infrequent occurrence.
- It is expensive. If SCU were to adopt it as a campuswide standard, it would cost roughly $30,000/year.
Links
E-mail this page
