Online assessments help The Arkansas Department of Workforce Education ensure quality vocational programs and provide metrics to support federal funding
There are hundreds of vocational programs in the state of Arkansas: classes at high schools, technical schools, and career centers, in subjects like auto mechanics, welding, or computer-aided drafting. Each year, tens of thousands students take these classes. To test their progress and document it for funding purposes, the Arkansas Department of Workforce Education uses Questionmark to create, schedule and administer those tests securely.
In previous years, the department administered its tests via paper and pencil. The data had to be reported to the federal government, but teachers often forgot to mail in students’ assessments. Students also often input their social security numbers incorrectly or filled out their answers in a manner that lost data. These errors meant that 20 percent of the data gathered was unusable.
Questionmark has allowed the department to gather far more data – nearly 98 percent – that is valid and reportable. Because online assessments require less work than paper-and-pencil assessments, it also saves the state money on personnel.
At the end of each vocational course, each student must take an assessment of 60 multiple-choice questions. With 33 courses and many students in multiple courses all across the state, the department administers 30-60,000 assessments each year.
The assessments show what students learned, of course, but the results from those assessments also help programs to safeguard their Perkins funding. The federal government requires, among other things, that educational programs which receive funding must report on the effectiveness of those programs. By demonstrating that the scores reported for a course improve every year, each program gets to keep the money coming in with which to make further equipment or curriculum purchases.
Questionmark Consulting Services helped the department build a Web site with a registration system that integrates with Questionmark. This allows the department to manage the hundreds of teachers at the vocational programs, who can schedule their classes and associate assessments with those classes. Teachers use the system to register students for classes and to schedule their assessments. All assessments are password-protected and delivered via Questionmark Secure—which makes it impossible to switch tasks or print assessments—so that students can only take each test once.
The consulting unit also created a custom-built error tracking system that helps teachers report problems with tests or scheduling, and customized reports that show how well students did on each question and topic area. These reports show instructors whether they need to adjust their teaching in a topic, and the department uses them to make sure that test questions are written clearly and coded with the right answers.
The biggest advantage of the custom system is that it automates both registration and the scheduling of tests. “We used to spend two weeks on registration and on creating 24,000 schedules,” says Karen Chisholm, program manager for the office of assessment and curriculum for the department. “The work that the consultants have done allows us to use a customized piece as well as the regular Questionmark Enterprise Manager simultaneously, and it’s going very well.
Questionmark has vastly improved the process of testing and data reporting that helps guarantee Perkins funding for schools all over Arkansas. “With Questionmark, we just have so much more flexibility,” says Keith Peterson, program advisor. “It lets teachers test students one, or two, or a whole class at a time, and it lets students catch up if they miss a test. It has boosted both the number of students we test and the amount of usable data, and it’s taken the human element out of test proctoring, meaning fewer mistakes, more data, and more funding.”