Posted By Doug Peterson
You’ve done your Job Task Analysis, created a competency model, and used it to develop a Test Content Outline (TCO). You’ve created well-written items that map back to your TCO. You’ve determined how many, and which type of, questions you need for each content area. You have avoided bias and stereotyping, and worked to ensure validity and reliability. You’ve developed your test directions for both the test-taker and the administrator. You’ve set your cutscore.
It’s finally time to deliver the assessment!
Here are some things to think about as you deliver your assessment:
If you’re using pencil and paper tests, you need to make sure the tests are stored in a secure location until test time. Test booklets and answer sheets must be numbered, and the test administrators should complete tracking forms that account for all booklets and answer sheets. Test-takers should be required to provide some form of identification to prove that they are the person who is scheduled for the exam.
Computer-based testing also needs to be secure. One way to increase security is to deliver the assessment in a testing center with a proctor in the room. If the test-takers are distributed across many locations, Questionmark offers Questionmark Secure, which locks down test-takers’ machines and doesn’t allow them to copy questions or switch tasks. Computer-based testing security can/should also include some form of identification and password verification.
Many times a testing organization will allow someone who fails a test to retest at some point. You also need to account for someone getting sick during the middle of a test, or getting an emergency phone call and having to leave. What if the power goes out in the middle of a computer-based test? You need to determine ahead of time what you will do in situations like these.
If the test is interrupted, will you let the test-taker resume the test (pick up where they left off) or take a new test? A lot of this has to do with the length of the interruption – did the test-taker have time to go off and look up any answers? This is not a consideration if your test doesn’t allow the participant to go back and change answers.
The problem with retesting is that the test-taker has already seen the questions. You should consider not providing individual question feedback if the test-taker fails the test, so that he/she doesn’t know what to go look up between tests. Most organizations require a waiting period between takes so that the questions will not be fresh in the test-taker’s mind.
A lot of the problem with retesting can be alleviated by creating multiple test forms (versions) with different questions. If a test-taker fails on their first attempt and wants to retest, you can give them a different form for the retest. At that point you don’t have to worry that they remembered any questions from the first attempt and went home to look up the answers, because they will be seeing all new questions. If you use multiple forms, you must ensure that the exact same topics are covered in the same depth, with questions having the same level of difficulty.
In the next post, we’ll take a look at controlling item exposure, limiting opportunities for cheating, and maintaining test integrity and ethics.
Posted By Doug Peterson