Now that’s a Good Question: Evaluating Our Online Exams
Concurrent Session 2
How well do our online exam questions assess our students’ capabilities? Investigate ways to evaluate the effectiveness of test questions. Explore alternatives to traditional multiple choice formats that maintain the advantage of automated grading.
Today’s students have instant access to Google and online copies of publishers’ test banks. This creates a challenge for online instructors to create exam questions that accurately assess a student’s abilities. How do you know if a question is really measuring what it is intended to? And, if it isn’t, how do you create questions that do effectively evaluate student abilities? This presentation will identify common traps for online test questions and look at the problems with standard publisher provided multiple choice tests.
Test item analysis is a tool instructors use to evaluate the effectiveness of individual test questions. Most Learning Management System’s item analysis tools provide difficulty and discriminant information. We will see how these can help you identify problematic questions. Instructors who study the item analysis of their tests often find that many of their multiple choice questions have a low or even negative discriminant. A negative discriminant means that students who received low scores on the test as a whole, did better on that question than did students who received high scores overall.
Multiple choice questions suffer from a high probability of guessing the correct answer which is often increased by poor distractors. Even teachers who understand how ineffective most multiple choice questions are, continue to use them because they are often provided with our textbooks and are easy to grade. Most LMS and testing systems provide alternative forms of questions that are easy to create and offer the advantages of automatic grading. Some of these are fill in the blank, numeric computation, matching, ordering, and multiple answer.
We will look at the pros and cons of these alternative question formats and look at transforming standard multiple choice questions into more effective alternatives.