Creighton University School of Pharmacy and Allied Health Professions

Scenario

Creighton University's Doctor of Pharmacy program has a strong assessment program designed to test students' mastery of course objectives and the program's ability based outcomes. The School of Pharmacy and Allied Health Professions uses Questionmark for regular multiple choice, numeric, essay, and hotspot questions for quizzes as well as end-of-semester course evaluations.

The school has used many methods of test delivery. Some quizzes and exams are administered in class to up to 100 students at a time. Some are administered online, both with and without proctors, and some are given in the computer lab to about 30 students at a time. In Pharmacy Calculations, for example, students are allowed to take the quizzes as often as they want. Each quiz is different from all others because questions are drawn randomly from a large, dynamic question bank that is constantly updated.

The Pharmacy program includes some students in a new web-assisted pathway, the first of its kind in the world. The 50 students using the web-assisted pathway spend a week on campus for orientation and computer training and return in the summer for lab work. All their regular courses are are administered online. The program includes a year of clinical rotations that may require a student in the web-assisted pathway to travel.

According to Assistant Professor Phillip Vuchetich, Pharm.D., the switch from paper-based exams has added to the value of assessments by turning them into learning tools as well as a means of measuring knowledge.

"Questionmark has enabled us to move to a mastery evaluation that includes both formative and summative assessment in Pharmacy Calculations and Pharmacokinetics," says Vuchetich. "Previously, these courses would have paper based exams which would either be scored by bubble sheet with multiple choice answers provided to students, or we would grade papers manually. We couldn't provide timely feedback on student performance, so this was only effective for summative assessment.

"Now, however, we include specific feedback along with the correct answer, which students see immediately after answering the questions, so they can use it to correct their mistakes for next time. In the Pharmacy Calculations course, the feedback is provided following the quiz or exam, but in Pharmacokinetics exams, a missed question provides feedback immediately with the correct answer before leading the students to a brief series of questions selected to identify the specific problem the student had. We've designed this feedback to be similar to what we show and tell students in class or in person when they ask questions."

"The feedback might say, 'Sorry, your answer was too low. The correct answer is: 123.45,' or 'Good job,' or 'You made a critical error - the correct answer is 123.45.' We are able to score critical errors as show-stopping events that actually bring the quiz or exam to and end and require that the student retake the assessment. Why? Because drug-related medical errors are likely to be a decimal point error - say 5 mg instead of 0.5 mg -that can be harmful or deadly in practice. By catching such an error end the test, we are ensuring that students do not perceive such seemingly minor mistakes as acceptable. They are not going to say, 'I did 90% of the problems right, but I just missed a decimal place.' They are going to understand the gravity of their mistake!"

Students appreciate these lessons, even though they can be painful at times.

One student emailed Vuchetich: "First of all, I have to tell you...Even though I found it completely frustrating at first, your method of giving your quizzes is truly ingenious! Don't get me wrong, it totally hacks me off when I get a critical error - especially when I've gotten all of the other questions correct. But it has forced me to be very, very, very careful and read the question several times and identify exactly what is being asked...But seriously, even though I had to take that last quiz like 7 or 8 times, I think I can do those equations in my sleep now! Hooray for me and good job on your part, Doc!"