Evaluating Exams
We assess our students on a regular basis, but we rarely stop to evaluate the assessment! Does your test assess student knowledge at the level you intended (e.g., descriptive, evaluate, apply, analyze)? Does it address the most important learning objectives of a unit or the course? Will the exam preparation and test itself encourage students to consolidate their knowledge and even extend their understanding of the material?
Evaluating tests—usually those tests with true-false, multiple choice, and short answer questions—includes investigating the test—e.g., measuring reliability and validity—and the individual questions—e.g., measuring difficulty and discrimination. The following descriptions offer more detail about each method of measurement:
- Reliability – How consistently does the test assess student achievement of the learning outcomes?
- Validity – How well does the test represent the knowledge or skills students need to achieve the learning outcomes?
- Difficulty – How hard is each question to answer? Calculated as the percentage of students who answered the question correctly.
- Discrimination – How well does each test question differentiate between students who perform well on the test—e.g., highest quartile—and those who perform poorly—e.g., lowest quartile?
Evaluating your tests does not mean you are trying to make the tests easier. It means you are making sure each test is fair, covers material students should have experienced through your class, and consistently assesses achievement of specific learning outcomes.
Learning management systems like Canvas (SCU's Camino) simplify the process quite a bit, by allowing you to review statistics related to the quiz results and even download a quiz item analysis.
- If analysis shows a question is very difficult, check the wording of both the question and response options. If the question itself is misleading, then you may choose to throw out that question. If a response option other than the correct one is partially true or correct in a different context, then you may choose to rescore that question and count the other response as correct or partially correct.
- In some cases, you may be drawing from pools or banks of test questions provided by a textbook publisher. Be sure to vet the questions before using them in a test, or evaluate those tests when you first use them.
Online Exams
Facilitating an online exam can present some unique logistical challenges. Below are a few tips to consider when creating an online exam.
Camino Quizzes
- Differentiate quizzes when you need to alter due dates or times for one or more students to let them take the same quiz at different times.
- Moderate quizzes to provide additional time or attempts for individual students who need to receive additional time. This feature also can be used to give additional time/attempts to students who inadvertently submit their quiz before finishing or experience technical difficulties while completing their quiz. You must ensure a quiz does not close before a student’s additional time ends. The "Available Until" date and time should be the time the last student has to complete the quiz.
- Beware fill-in-the-blank questions, as student responses must match the correct answer character-for-character in order to be marked as correct.
- Configure quiz settings to configure when students can view correct quiz answers.
- Essay questions can be impacted by network connectivity issues that cause student work to be lost. You can ask students to draft responses in a separate document and paste into the quiz, or use a file upload question instead of an essay question.
For a quick guide on summative assessments, check out the infographic from the Scholarly Teacher below:
Evaluation tips for efficiency and effectiveness
While grading faculty often tend to edit students’ answers instead of providing constructive feedback. What is a good balance between a few words that often leave students confused and extensive corrections that overwhelm us with repeated comments and endless grading? There is also a balance between trying to provide important and constructive feedback with not overwhelming yourself with workload. Therefore, reflect on ways that you can increase efficiency with still communicating constructive feedback effectively.
Here’s one way to do it after an exam:
- Create a list of exam comments based on anonymous students’ answers.
- Circulate the list with all your students so that everyone sees weaker and stronger responses along with your feedback.
- Make this list their future study guide for the final exam, and save time on both ends.
- Make sure that you clearly explain your decision-points from the exam to help students understand why the answer was what it was. This can be done as a conversation with the class or even in small discussion groups.
- Make this list their future study guide for the final exam, and save time on both ends.
Additional Resources
Brame, C. (2013). Writing good multiple choice test questions.
Weimer, M. (May 10, 2013). Exams: Maximizing their learning potential. Faculty Focus.
University of Wisconsin-Madison Tips for grading efficiently
Page authors:
Dr. Kevin Kelly, Lecturer at San Francisco State University
Brian Larkin, SCU Instructional Technology Manager
Sofia Kotsiri, Faculty Associate with the Collaborative, and Lecturer in Economics
Last updated:
February 20, 2024