Sections:
Evaluating Exams
Online Exams
Additional Resources

Evaluating Exams

We assess our students on a regular basis, but we rarely stop to evaluate the assessment! Does your test assess student knowledge at the level you intended (e.g., descriptive, evaluate, apply, analyze)? Does it address the most important learning objectives of a unit or the course? Will the exam preparation and test itself encourage students to consolidate their knowledge and even extend their understanding of the material?

Evaluating tests—usually those tests with true-false, multiple choice, and short answer questions—includes investigating the test—e.g., measuring reliability and validity—and the individual questions—e.g., measuring difficulty and discrimination. The following descriptions offer more detail about each method of measurement:

Reliability – How consistently does the test assess student achievement of the learning outcomes?
Validity – How well does the test represent the knowledge or skills students need to achieve the learning outcomes?
Difficulty – How hard is each question to answer? Calculated as the percentage of students who answered the question correctly.
Discrimination – How well does each test question differentiate between students who perform well on the test—e.g., highest quartile—and those who perform poorly—e.g., lowest quartile?

Evaluating your tests does not mean you are trying to make the tests easier. It means you are making sure each test is fair, covers material students should have experienced through your class, and consistently assesses achievement of specific learning outcomes.

Learning management systems like Canvas (SCU's Camino) simplify the process quite a bit, by allowing you to review statistics related to the quiz results and even download a quiz item analysis.

If analysis shows a question is very difficult, check the wording of both the question and response options. If the question itself is misleading, then you may choose to throw out that question. If a response option other than the correct one is partially true or correct in a different context, then you may choose to rescore that question and count the other response as correct or partially correct.
In some cases, you may be drawing from pools or banks of test questions provided by a textbook publisher. Be sure to vet the questions before using them in a test, or evaluate those tests when you first use them.

Online Exams

Facilitating an online exam can present some unique logistical challenges. Below are a few tips to consider when creating an online exam.

Camino Quizzes

Use these steps to differentiate your quiz if you need to alter due dates or times for one or many students. This is for students who will be taking the same quiz but at a different day or time.
Use these steps to moderate your quiz if you need to provide additional time or attempts for an individual student. This is for students who receive additional time, for students who inadvertently submit their quiz, or for students who need additional time or attempts due to technical difficulties occurring during the quiz. Make sure the quiz does not close before the additional time for a student is up. The "Available Until" date and time should be the time the last student has to complete the quiz.
Avoid fill-in-the-blank questions. Students are provided unlimited options for entry, but their answer will have to match the answer character-for-character in order to be marked as correct.
Make sure to use settings so that students cannot see the correct answers after the quiz. A good strategy is to create a date range to display the correct answers after every student has taken the quiz, set a start date in the Show field, and set an ending date in the Hide field.
Set a due date and time for the quiz. Students will see the time for the time zone of your class, and they can change that setting for their account.

Essay Questions

Within Camino quizzes, you can include essay questions. Keep in mind, though, that if students spend a long time writing in the text box and then experience network connectivity issues, they could lose their work. There are two safeguards for this. You can recommend students write their responses in a separate document and then paste it over to the quiz, or you could create a File Upload question where students upload a PDF, Doc, or docx response.
If you want to let students choose from a list of essay prompts, the best way to do this is to set up the essay as an assignment in Camino. This way, you can post the prompts and your grading criteria, and students can respond in a Google Doc or Word file. If you set up multiple essay prompts in a Camino quiz and ask students to only respond to a portion of them, they will automatically lose points for not filling out all the prompts.

LockDown Browser and Respondus Monitor

See this guide for detailed information about using LockDown Browser and Respondus Monitor.
Share the software link and Respondus guide with students well before the actual test so that they have time to prepare.
Give students a low-stakes practice exam before the high-stakes assessment for them to work out any tech issues.
Provide alternative assignments for students who are unable to use the software.
Take a practice exam (available in the “Keep Calm and Carry On” Camino course) to experience the software for yourself.

For a quick guide on summative assessments, check out the infographic from the Scholarly Teacher below:

View directly, or right-click to save a copy.

Additional Resources

Brame, C. (2013). Writing good multiple choice test questions.