I promised in an earlier blog entry to pass on my understanding of research in educational psychology about the unmediated or direct benefits of questioning, i.e., how answering questions helps people learn. I’ve recently read a 2008 paper by Butler and Roediger from the Memory Lab at Washington University in St. Louis, Missouri (see here for the 2008 paper and here for a 2010 review paper including the graph below), which includes some fascinating information on how multiple choice quizzes directly aid learning.
The researchers divided students randomly into four groups as follows
- Study a subject, no quiz
- Study a subject, take a multiple choice quiz, no feedback
- Study a subject, take a multiple choice quiz, feedback after each question
- Study a subject, take a multiple choice quiz, feedback at the end of the quiz
They then tested all the groups a week later and got the results below.
As you can see, the students who had taken a quiz (or test as the authors describe it) got better results on average than those who hadn’t taken a quiz. This is expected due to the general principle that answering questions gives retrieval practice, which helps you to be able to recall things later and so helps learning. This is similar to results I’ve blogged on elsewhere.
However, what is interesting on this study is that on multiple choice quizzes, there is the potential danger that students will choose the wrong answers and so think they have retrieved information which is in fact wrong. What this study showed was that if you give feedback on the quiz, then this improves learning further, as you can see in the graph above. Interestingly, although you might think that immediate feedback right after the question is best, this wasn’t the case in this example. Quizzes with feedback delayed until the end of the assessment gave better results than those with feedback after each question. The authors postulate that a slight gap in giving the feedback allows the incorrect concept to dissipate before the feedback is given and also gives spacing in time, which helps learning.
My summary of understanding from this research:
- Giving a quiz after learning will help retention, as it gives recall practice
- Giving feedback helps improve retention, particularly in multiple choice quizzes where there is a danger of learners choosing wrong answers and thinking they are right
- Feedback is better at the end of the quiz, not after each question
For more information on the research, see Professor Roediger’s publications page at http://psych.wustl.edu/memory/publications/.
One interesting issue this raises is that it’s common in certification exams not to give feedback, to retain the confidentiality of the questions by not repeating them, and because certification aims at measuring rather than learning. What this research shows is that if you want to help your successful and failing candidates learn, then you could consider feedback in some form.
Here’s a question to allow you to practice retrieval on the subject of this blog:
Should you give feedback on multiple choice quizzes after each question or at the end of the assessment?