I have taken many kinds of exams over the course of my learning journey and the ones that probably terrorize me the most are Multiple Choice Questions. Why? Because I have taken them in the negative marking format where I gained 4 points for every right answer and lost 1 point for every wrong answer. Add a cup of competition to such as scenario and you will get fear and uncertainty and often double thoughts about every answer because you really don’t want to loose points. Thankfully, such a format wasn’t used in school assessments, and I do not plan to use such a format in my practices either. 🙂
In this article, I will give an overview of the different kinds of selected response items that an educator can integrate in their assessments practices.
Select Response items are those where we choose a response from the given responses. Thus, the idea is not to articulate the answer but to select one of the given answers.
There are three kinds of selective response items, as shown on the left here. As I progress through each of the types, I will mention their structure, advantages and weaknesses and link them back to the concepts of validity and reliability, as well as summative and formative assessments that were covered in the previous article (Assessments 101).
Alternate Responses or True/False Items or Binary Response Items
Such kind of selective responses are constructed from statement within the text. There are only two options for the statement. The most common form is to mark the statement as true or false though other options could be to mark the statement as an opinion or a fact based on the context of the exercise.
These offer a quick way to review key points and are easy to make because they are usually adopted from the text for the course. I often use binary response items in formative quizzes on Kahoot for this very reason.
Advantages of Binary Response Items:
- They are time-efficient because they easy to write as well as score.
- Related to this, a great of information can be sampled using such questions because it does not take too long to complete the test.
- Scoring is objective in nature, i.e, any biases that the teacher might have (examined in a forthcoming article) do not affect the score.
Disadvantages of Binary Response Items:
- The chances of guessing is high as one of the answers is correct. This makes the items unreliable as the answers may be from guessing.
- Such items can only be created by lower levels of the Bloom’s taxonomy, i.e., remembering and understanding.
A Penn State University article offers some insights into writing good alternate response questions.
Matching Items
I remember matching items from elementary and junior high. I do think I saw their use in my high school and later education. However, they are a great assessment items type to have because they are able to assess relationships and connections between two groups.
Advantages of Matching Items:
- They are time-efficient: they easy to make as well as score.
- A good of information can be sampled using such questions about relationships and related concepts.
- Scoring is objective in nature.
- Guessing is generally hard to accomplish.
Disadvantages of Matching Items:
- Due to their unique setup where the stem faces the responses, it might be hard for students, especially those who struggle with reading comprehension, to create the connections. They might not be able to organize their thoughts to match the two sides.
- Matching items only test remembering and understanding on Bloom’s taxonomy.
One best practice that applies to building matching items is to ensure that the stems/premises and responses/answers are homogenous in nature. By mixing relationships between planets and parts of the plant, for instance, we will not be writing good matching items because a student with testwiseness would be able to eliminate options and get to the right answer without knowing it.
Multiple Choice Items
Multiple choice items have 3 or more responses to choose from. I recently spent some time creating Multiple Choice items on higher levels of Bloom’s taxonomy and I would admit it was one of those tasks that put me in the Zone of Proximal Development — I knew exactly the kind of questions I had to make but I struggled because I had to create them from scratch. I have answered many Multiple Choice items for Math but creating was a totally different story.
The items with the source are also called interpretive exercises according to McMillan (2011).
Advantages of Multiple Choice Items:
- Like the other Selective response types, they are time-efficient: they easy to make as well as score.
- It is possible to test higher order thinking skills with these questions.
- Scoring is objective in nature.
- Due to the large number of responses, guessing is hard to accomplish if alternatives are written well. Also, the diagnostic power of the items increase because I can find out common misconceptions and mistakes that students are making, identifying concepts that might not be clear.
- They allow for a broad sampling of knowledge.
These advantages of Multiple Choice items allow them to be more valid and reliable as explained by Carmen (2013):
Reliability: The reliability is enhanced when the number of MC items focused on a single learning objective is increased. In addition, the objective scoring associated with multiple choice test items frees them from problems with scorer inconsistency that can plague scoring of essay questions.
Validity: Because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the assessment.
Disadvantages of Multiple Choice Items:
- They are time-consuming to write and it generally harder to come up with more than 2 good distractors. Since so much effort is required to write them, I suspect this is a good reason to use these items for summative assessments, often in preamble to constructed response items to get the remembering and analyzing concepts out of the way where the student does not need to show their thinking.
- Writing high order thinking questions is hard. With Math I found that the trick was to incorporate a source in the question.
With regards to the source, some best practices include ensuring that it only shows information that is vital to solving the following questions. Since sources might be time-consuming to read, they should be as brief as possible and there should be at least a couple of questions related to the source. Another best practice is to make it familiar, but new to the student.
There are specific practices related to the stem and alternatives that are explained well by Camen (2013) with examples. A good resource to learn about writing good items is Mullins and Martin (2013). They have a fantastic section about creating distractors.
Best Practices in Writing Items (common to all types)
- Statements should be phrases in a positive manner, avoiding double negatives.
- Each sentence and phrase should be expressed in a clear and concise manner, removing redundancy and extra words. It should be checked for grammar and spelling.
- Absolutes should be used with caution.
- Items should be free from pop-culture references that students might not be able to understand. They should be fair and equitable to all students (see Assessment 101 for description of fairness in assessments).
- Key words such as best and least should be emphasized.
Conclusion
All selective response item types that we reviewed have many commonalities. They mostly assess lower levels of Bloom’s taxonomy though we can create Multiple Choice items for higher levels with some work. All types are objectively scored as there is a set answer key and can be completely by the students in less time as compared to constructed response items. These types influence student learning by encouraging students to remember, recall, use concepts and interpret given information. Guessing is a major drawback of these item types because the answer is present and just needs to be chosen.
Each item type has its own sweet place in the learning world. While selective response types are quick to administer, generate a large sample of information about student knowledge and can be adapted to higher levels of Bloom’s taxonomy with extra effort, they do not give a glimpse into the articulation ability or the thought process that the student engaged in. For this, we can use constructed response items… more on those in the coming weeks!
In what situations do you use selective response items?
Share your views on twitter with @a_teachers_hat or @_armedwithabook.
References:
Brame, C., (2013) Writing good multiple choice test questions. Retrieved October 10, 2018 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/.
McMillan, J. H. (2011). Classroom assessment: principles and practice for effective standards-based instruction 5th ed. Boston: Ally & Bacon.
Mullins, I. V. S., & Martin, M. O. (2013). Item writing guidelines. Retrieved October 10, 2018 from https://timssandpirls.bc.edu/publications/timss/T15_item_writing_guidelines.pdf
Be First to Comment