Posted By Doug Peterson
Writing good items and putting together valid and reliable assessments can take a lot of time and cost a lot of money. Part of an assessment’s reliability and validity is based on the test-taker not knowing the items ahead of time. For these reasons, it is critical that item exposure be controlled.
This starts during the development process by requiring everyone who develops items or assessments to sign a confidentiality agreement. Developers’ computers should, at the very least, be password-protected, and you should consider data encryption as well.
Care must be taken to prevent question theft once an assessment is assembled and delivered. Do not allow overly generous time limits, which would provide time for a test-taker to go back through the assessment and memorize questions. If your assessment is delivered electronically, consider not allowing backward movement through the test. Be very careful about allowing the use of a “scribble sheet”, as someone might try to write down questions and sneak them out of the test center: be sure to number all scribble sheets and collect them at the end of the assessment.
Computer-based testing makes it very easy to utilize random item selection when the assessment is delivered. While this does mean having to develop more items, it cuts down the number of times any one item is delivered and helps to reduce cheating by presenting different questions in a different order to teach test-taker.
It is critical to track the number of times an item has been delivered. After a certain number of deliveries, you will want to retire an item and replace it with a new item. The main factor that impacts how many times an item should be exposed is whether the assessment is high-stakes or low-stakes. Items on a high-stakes exam should have a lower maximum number of exposures, but items on a low-stakes exam can have a higher number of exposures.
As long as there have been tests, there have been test-takers who try to cheat. Make sure that you authenticate each examinee to ensure that the person who is supposed to be taking the exam is, in fact, the person taking the exam. Testing centers typically prohibit talking, using notes, and using cell phones during tests. Maintain a minimum amount of space between test-takers, or use carrels to physically separate them.
Test administrators should walk around the room during the test. Unauthorized personnel should not be permitted to enter the room during the test, and the administrator should not leave the room for any reason without first bringing in another administrator.
Computer-based testing presents its own set of security challenges, especially when testing is permitted outside of a secure testing center (e.g., in the test-taker’s home). Questionmark offers the Questionmark Secure client, which locks down test-takers’ machines and doesn’t allow them to copy questions or switch tasks.
Computer-based testing security can/should also include some form of identification and password verification. Additionally, in the last few years, technology has become available that allows for the remote monitoring of test-takers using built-in laptop/tablet cameras or small desktop devices.
Click here for links to a complete listing of posts in this series.
Posted By Doug Peterson