A lot of you write test questions for online training (or even for paper-based training).
Maybe you’re doing it with an e-learning authoring tool, such as the ones from Articulate, Adobe, or Lectora, or maybe you’re doing it with quizmaking tool built into your learning management system (LMS). Or maybe with pencil and paper.
However you’re doing it, you may sometimes find yourself wondering about the best practices for writing standard question types. (By the way, instructional designers often use the wonky phrases “assessment” for a test and “assessment item” for a question within a test.)
We’ve got a few of those best practices for you below. Hope this helps you with your question writin’. 🙂
General Tips for Writing Test Questions
Let’s cover some general tips for writing questions (“assessment items”) before we zero in on multiple-choice questions.
We’re going to separate these into Do’s and Don’ts.
What To Do When You Write Questions: General Tips
Here are some things that you SHOULD DO when you’re writing questions for tests.
Do Create Questions for All of Your Learning Objectives
Make sure your questions assess learning objectives and nothing but learning objectives.
And make sure your questions cover all of the learning objectives (it can be easy to miss one or more sometimes).
You can write more than one question per learning objective.
Don’t know what a learning objective is? Here’s a good guide to learning objectives.
Do Create Questions that are True Assessments of Your Learning Objectives
In workforce training, most training is intended to teach people to do something. As in, perform a task or skill.
And, in most cases, your goal for creating the training isn’t to have the employee later (a) demonstrate knowledge on the job or (b) correctly answer a multiple-choice question.
So think hard before you sit down to write a multiple-choice question. Is this really assessing the workers’ ability to satisfy the learning objective, or are you just working quickly in default mode and writing a multiple-choice question when a real skills performance might be more appropriate?
I’m not saying there’s never a time to use multiple-choice questions. But I am saying they tend to be overused, and there may be better options.
Do Write Questions that Anticipate Common Misunderstandings/Errors
Now, this is different than a trick question. And it’s OK and even good to write questions like this.
These kind of questions can help you identify people who only “kind of” understand, or understand superficially, or have a common misunderstanding that needs to be corrected.
Do Stick to One Point per Question
Write questions that address one point per question.
Avoid writing questions that attempt to assess multiple points.
Do Include Feedback
Find a way to include feedback to inform the employee if they got the answer correct or incorrect, and what the correct answer is.
This can be easier to do with online systems, and is a nice benefit of the kind of automation you can get with online learning management systems, but you can do it after a test is turned in if you’re grading by hand.
Do Review and Edit Questions
Don’t just write your questions and let ’em out in the wild. Review all of your tests and test items carefully before you deliver them to employees.
Check for poorly written questions that are confusing or vague; check for typos, spelling and grammar errors, and any other miscues you don’t want to be associated with; check the reading level to see if it’s appropriate for your testing population; check to make sure the questions don’t include a bias, putting one sub-group of employees at an unfair advantage or disadvantage; check for material that’s offensive, insulting, or needlessly controversial or disturbing.
If you can, have another person (or even better, a team) review them as well–it’s always hard to review your own work. Then maybe see if you can get a small group of employees to “beta test” your test as a pilot.
What NOT To Do When You Write Questions: General Tips
Now let’s look at some things you SHOULDN’T DO when writing questions for tests.
Don’t Write Trick Questions
Don’t write trick questions.
You’re creating assessments and writing assessment items (“questions”) because you want to see if employees know something.
You’re not writing a test item to see if employees are especially skillful at taking tests or at recognizing trick questions.
So, if your goal is to determine if employees know something, and not to see how clever they are at sniffing out trick questions, don’t write trick questions. No matter how tempting it may be.
Don’t Write Questions that Test Only Recall
In many cases, it’s easier to write a test question that tests an employee’s ability to recall material than it is to write a question that tests their ability to apply information.
Avoid writing questions that focus only or too much on recall/recognition.
(For more context on recall/recognition, application, and other forms of learning, see this article on Bloom’s taxonomy.)
Don’t Help Employees Identify Correct Answer Based on Tips from Grammar
You can sometimes make it easier for an employee to answer a question correctly just with the grammar you use.
For example, if the question is phrased in a singular manner, and all answer options are plural except one answer option that’s singular, that singular answer option is probably the correct answer.
Or, for another example, if your question stem ends with “an” and the four answer options begin with “cat,” “dog,” “leopard,” and “otter,” employees will know that the correct answer is “an otter.”
Review your questions to make sure you’re not tipping your hand and making it easy for employees who don’t know the correct answer to use grammar tips to identify the correct answer.
Don’t Write Questions That Provide Answers to Other Questions
Try to avoid writing a question that includes information that makes it easier for employees to answer a second question.
Don’t Write a Question Because It’s “Easy” to Write or Grade
If you want to create effective assessment items (“questions”), then do it. Don’t take the easy way out and create an assessment that doesn’t do what you want it to do.
We all get tired sometime; I get it. It can be tempting to create a question in a certain format just because it’s easier to write. Or, thinking down the road, because it’s easier to evaluate/grade.
But don’t do that; don’t give in to your weaker moments. Be a training superstar and write the best question you can–one that truly assesses whether or not employees understand the content.
Don’t Write Questions That Are Phrased with Double Negatives–and Try to Avoid Negative Phrasings Too
Avoid writing questions that are phrased with double negatives. For example, a question that asks “Which of the following is NOT unnecessary?” has a double negative (“not” and “unnecessary”) and can be confusing. Rewrite a question like this actively, to something like “Which of the following is necessary?”
You should always avoid double-negatives, as we just said. But in addition, you may want to be careful about questions that include negatives of any sort, and see if you can rewrite it in a more active manner. For example, “Which of the following tools should not be in a hot work area?” could be rewritten actively as “Which of the following tools is forbidden in a hot work area?”
Tips for Writing Multiple-Choice Questions
Now let’s give some tips that are specific to multiple-choice questions.
Parts of a Multiple Choice Question
Let’s begin by talking about and naming the different parts of a multiple-choice question. We’ve written a sample multiple-choice question below and have labeled its different parts.
- Stem: The base of question (in this case, “What color is the sky?”).
- Answer options: The possible answer choices. This includes the correct and incorrect answers. In this example, it’s A, B, C, and D (or red, orange, blue, and purple).
- Correct answer: The answer option that is correct. In this example, it’s C, or blue.
- Distractors: All of the answer options other than the correct answer. In this example, it’s A, B, and D, or red, orange, and purple.
Now that we’ve got our terms down, let’s look at some guidelines for creating each part of a multiple-choice question:
Tips for Writing the Stem of a Multiple-Choice Question
Here are some tips for writing the stem:
- Keep the stem short and simple.
- Keep the wording precise. Don’t write something that’s vague, ambiguous, or confusing.
- Try to write the stem as a complete question (such as “What color is the sky?”) instead of fill-in-the-blank (“The sky is _____.”) when possible.
- If you do write a fill-in-the-blank-style question, put the “blank” space at the end of the stem and not in the middle when possible. For example, write this (“The author who wrote Hamlet is ________.”) instead of this (“The writer _____ wrote Hamlet.”).
- Put any wording that’s repeated in every answer alternative into the stem, so it only has to be written once.
Tips for Writing the Answer Options of a Multiple-Choice Question
Here are some tips for writing the answer options:
- All answer options should be “plausible.” This means they are all reasonable alternatives. Don’t write answer options that obviously can’t be correct.
- Write all answer options in a simple, clear manner.
- Aim for at least three answer options per question.
- There’s no single “correct” number of answer options to include in a question. Four or five seems to be a standard.
- It is not absolutely necessary to always have the same number of answer options for each question. Try to remain consistent when possible, though. If you need to add one or more answer options beyond your standard, that’s OK if there’s a good reason but don’t do it without a good reason. If you can’t seem to come up with enough answer options, think hard and try to create one or more plausible one. But if you can’t find enough plausible answer options, it’s better to go with fewer answer options than to create non-plausible answer options.
- Avoid writing answer options like “B and C”.
- Avoid using “All of the above” and “None of the above” as answer options. “All of the above” tends to be the correct answer a large number of times when it’s used, and so it helps employees guess the correct answer. Plus, if an employee knows that just one of the answer options is incorrect, the employee will know that “All of the above” is incorrect, too. And if “None of the above” is the correct answer, the question item doesn’t tell you if the employee knows the correct answer.
- Use the words the words all, always, and never with caution. These words tend to indicate an incorrect answer.
- Be careful when using words like usually, generally, or typically. These words tend to indicate a correct answer.
- Try to write all answer options in a similar format. If you know your grammar a bit, this includes writing your answer options in a parallel structure. If you’re not familiar with “parallel structure,” that link you just passed should help.
- Try to keep all answer options to a similar length.
- Be especially careful not to write the correct answer in a way that’s different than the other answer options (for example, notably longer–this is common).
- When possible, put the answer options in a logical order (alphabetic, numeric, some other logical order) or randomize the order of the answer options.
- Avoid answer options that overlap.
- Avoid writing more than one answer option that is arguably correct (unless it’s a multiple-response question that accepts multiple answers).
Tips for Writing the Correct Answer of a Multiple-Choice Question
Here are some tips for writing the correct answer:
- Don’t copy the correct answer word-for-word directly from the training materials. Instead, paraphrase the correct answer so it means the same thing in slightly different words.
- Double-check to make sure that the correct answer is written in a similar fashion as the other answer options. Correct answers are often written in a different manner (for example, they’re often longer) and as a result they can help employees guess the correct answer.
- “Mix up” the correct answers, so that they’re randomly distributed amongst A, B, C, and D not always/often the same value–such as C. For example, in a 10-question test, these correct answers are mixed up well (adcbbadbca) and these correct answers are not mixed up well (acbcdccbcc). Also, watch for obvious patterns (abcdabcdab). It may help you to know that question writers tend to make B and C the correct answer most often.
- Related to the last point (and to our earlier point about trick questions), don’t try to play “head games” with employees by doing things like having the correct answer be C for ten questions in a row. Again, you’re trying to create an effective assessment of workforce training (and learning), not a psychology test.
Tips for Writing the Distractors of a Multiple-Choice Question
Here are some tips for writing the distractors:
- Write distractors that could be correct (learning experts call this making the distractors plausible). Don’t write nonsense answer options. This can be hard, but dig deep and put in the work to come up with good distractors.
- Don’t write joke distractors. There are other times to be funny.
- Don’t include distractors that cover material that you can’t be confident all employees are familiar with as a result of the training. This can create a biased question that allow some employees to answer the question correctly more easily than other employees because of something they are familiar with from outside of the training.
Multiple-Choice Questions in Non-Traditional Formats
Remember that a lot of eLearning authoring tools (Articulate, Captivate, etc.) allow you to create what amounts to a multiple-choice question in non-standard formats. For example, maybe the learner will click one of three doors, or select one of four packages.
Always remember you can get more creative with these, and doing so may make the question more authentic and/or more engaging.
Conclusion: Writing Multiple-Choice Questions for Tests
Well, that’s what we’ve got for you.
What are your own thoughts? About writing tests and test items in general? About writing multiple-choice questions?
If you found this interesting, you can also enjoy this other article about workforce training assessment, which introduces the topics of reliability and validity and covers more stuff, too.
Also, stay tuned for a future blog post on the topic of performance assessments–assessments that ask a worker to perform a real-world job task. That’s coming, we think.
And don’t forget to download the free Guide to Effective Manufacturing Training below while you’re here.