Dallas Baptist University: Aggregating Data from Online and Paper-Based Assessments
Dallas Baptist University is a private, liberal arts university in Dallas with extension sites in the Dallas-Fort Worth area and more than 5,200 students in the arts, sciences, and professional studies at undergraduate and graduate levels.
The university began using Questionmark to create an online assessment that determined placement for students in music classes. Using audio files integrated with multiple-choice questions and Adobe Flash, the test assessed students' levels of music knowledge. After listening to a short piece of music, students then identified major and minor chords, musical notes, pitches and other musical concepts.
Use of Questionmark at DBU has spread to nearly all departments as an alternative to paper-and-pencil tests. In order to aggregate the data from both types of assessment delivery, the Online Education department uses the Printing and Scanning feature in Questionmark. Students use Scantron sheets to mark their answers, then the answers are scanned so that the response data can be read and uploaded by Questionmark.
"The ability to aggregate data is important, because we do comparison studies to make sure that students are having comparable learning outcomes whether they take courses that are delivered online, traditionally, or as a hybrid of the two," says Kaye Shelton, Dean of Online Education.
Currently, about 25 professors use Questionmark to deliver assessments, and a few hundred students take those assessments each semester. Kevin Bock, an online course developer in DBU's online education department, says that number is likely to grow quickly as more professors see the value of delivering assessments online.
One feature that is likely to help with that expansion is the Questionmark Blackboard Connector, which connects Questionmark to the Blackboard Learning System™ and allows students to move back and forth between the two easily. When students log on to Blackboard, they can link to and take required Questionmark assessments, and receive immediate feedback on their performance. They can also see their scores in the Gradebook, and when they view their grades, the system brings up a Coaching Report for that assessment.
"It's a pretty seamless experience for the students, and they never even have to leave Blackboard," Bock says.
Shelton's department has also begun using Questionmark to gather data on learner outcomes. Instead of demonstrating that they are graduating a certain number of students, accrediting agencies want schools to show that students who graduate have learned certain skills. So DBU wants to assess its graduates’ core competencies, such as critical thinking, reading, writing, mathematics, and others.
Shelton says that Questionmark will be the key for providing additional assessment data for its annual reports and reaccreditation documents. Many of the student learning outcomes for which DBU gathers data are assessed by the Measure of Academic Proficiency and Progress standardized test. But by adding specific "learner outcome" questions to tests in courses throughout the curriculum, DBU is able to aggregate data across the university that helps show that graduates possess desired competencies.
Course developer Kevin Bock uses topic structure with Questionmark to tag each assessment question with one of the learner outcomes it measures, which allows Shelton's department to show specific data on each learner outcome. In order to gather the data required, they export assessment data to ASCII to create customized reports.
"Like many colleges, we had to figure out how we were going to deliver the data that we needed for faculty to determine the degree to which learned outcomes were being met," Shelton says. "Questionmark makes it easier for us to aggregate data from student assessments and show that we're graduating students with competencies achieved."