multiple choice

Kaizen: A Gaming Platform for Academic and Patient Education

Carolynn Thomas Jones and her colleagues presented a webinar on Kaizen, a gamification platform for teaching and learning. Kaizen has been used in academic programs in public health and nursing, including our own MACPR courses, to teach students about clinical research quality management and Good Clinical Practices (GCPs). Presenters for this session shared their experience with Kaizen and their development process for a game-manager user manual (their “field guide” to Kaizen). If you’ve been considering integration of an element of gamification in your course, this platform might be the tool you need!

View the webinar recording to find out more!

Assessing the Quality of Multiple Choice Questions

I am often asked about how one knows whether a multiple choice question is “good” or not. Expertise in the field always makes the final decision, but there are guidelines and statistics that can provide very helpful support in writing and refining multiple choice questions.

Nikole Hicks, PhD, RN, CNE, wrote a Fairness of Items tool (FIT) to guide writing and assessment of multiple choice questions. Read more about her development of this tool. I have her permission to share it with OSU College of Nursing faculty, so email me (Joni) to obtain a copy.

If you are somewhat familiar with statistics and need a quick guide to quiz item analysis, refer to this PDF from Anne Schoening, PhD, RN CNE.

A more detailed explanation of item analysis of quiz questions is presented in this article McGahee and Ball (2009). You can access the full text of the article through the OSU Health Sciences Library.

As always, please consult with the your instructional design experts on the CON-IT team if you need additional assistance with question writing and evaluation.

Multiple Choice vs. Multiple Answer

The “Quizzes” tool in Canvas is a easy way to allow students to take a quiz or survey online. Not only is this feature easy to use, it also comes with a large number of  features to customize a quiz to fit your exact needs. You can make it a quiz or a survey, a graded or a practice quiz, and even add a variety of question types such as matching and true/false.

With 12 different question types to choose from, there may be some confusion as to which question type does what. Two very similar sounding types are “Multiple Choice” and “Multiple Answer.” Although they sound the same, they produce very different types of questions. Below is a tutorial on how each of these question types work.

Multiple Choice vs. Multiple Answer tutorial

Go to the quizzes tab in Canvas and select “+ QUIZ.”

 

Select the “Questions” tab.

 

Select “+ NEW QUESTION.”

 

Here you can select which type of question you want. In this tutorial, we will select “Multiple Choice.”

 

Type your quiz question into the text box.

 

Type your possible answers into the texts boxes.

 

Select “UPDATE QUESTION.”

 

Select “+ NEW QUESTION” again. This time instead of selecting “Multiple Choice,” we are going to select “Multiple Answers.”

 

Note the warning above the text box. “This question will have a checkbox next to each answer and the student must select ALL the answers you mark as correct.”  Do not select this question type if you only have one correct answer in your question. For questions with only one correct answer, use “Multiple Choice.” For questions where you want the student to select more than one correct answer, use “Multiple Answer.”

 

Type your quiz question into the text box.

 

To turn a “Possible Answer” into a “Correct Answer,” click the arrow next to the answer. Notice if you were in a “Multiple Choice” question, the “Correct Answer” would simply move to that question row. You cannot have more than one correct answer in “Multiple Choice.”

 

Type in your possible answers into the text boxes. Notice since a net and volleyball are both needed for a volleyball game, both answers are marked as correct.

 

Select “UPDATE QUESTION.”

 

This is what the quiz questions will look like from the student perspective. Notice “Multiple Choice” questions have circles (also called radio buttons) next to the answers while “Multiple Answer” questions have squares (also called checkboxes).

If a student sees a question with squares (checkboxes) by the answers, they will assume more than one answer choice is possibly correct and can choose more than one answer. This circle and square convention holds true across quizzes and survey created in many different applications, including Qualtrics, SurveyMonkey, and the NCLEX.  This is why it is important to only use “Multiple Answer” questions on your quizzes if you want the student to believe that more than one answer may be correct.

 

Notice “Multiple Choice” questions will only allow the student to select one answer while “Multiple Answer” questions will allow the student to select several answers.

Fairness of Items (FIT) tool for multiple choice exams

Pearls of wisdom from the STTI 43rd Biennial Convention

My series of posts on the recent STTI Biennial continues with a summary of Nikole Hicks’ presentation titled,

“Are Your Multiple-Choice Tests “FIT”?  Using the Fairness of Items Tool (FIT as a Component of the Test Development Process”

In short, Nikole did an exhaustive review of the literature to learn about best practices in test item writing with a focus on nursing education.  She distilled the guidelines into 38 criteria to determine whether a single multiple choice test question is fair and unbiased.  She rigorously tested her FIT tool with nursing faculty and found it to be valid and reliable.  Read the full description of her study background, methodology, results and conclusions on the STTI conference web site.

Her list of 38 criteria can be used to evaluate a single multiple-choice test question, or they can be used to guide test question writing.  They are divided into four categories:

  • evaluate the stem
  • evaluate the options
  • linguistic/structural bias
  • cultural bias

The criteria include recommendations regarding how many distracters to include, words and phrases to avoid, and page formatting, among many other things.  She recommends that nurse educators use the FIT tool to write original questions and revise publisher test bank questions to improve student success and better prepare students for licensure exams.

I found Nikole’s presentation to be very interesting, and the tool has the potential to be very useful in ensuring fairness of multiple choice exams.  Many thanks to Nikole for doing the work of combing through the extensive body of literature to condense item-writing best practices into a practical set of guidelines we can really use.   I have permission from Nikole to share the tool with OSU College of Nursing faculty, and I plan to offer a Flash Friday session on the tool in the spring semester.