Posted by: John Kleeman, Founder and Executive Director
The lockdown has caused a big rise in online testing as face-to-face training has not been possible.
Companies and training outfits are moving from face-to-face training to online training and need post-course tests. Universities and colleges are giving exams to students at home. Certification tests, previously taken at test centers, are now being taken online with remote proctoring via video.
For most, the experience has been positive. In recent weeks, we’ve helped a variety of organizations move over to the new environment. Some may change their practices forever. In April and May 2020, we’ve delivered well over three million assessments on Questionmark OnDemand, nearly 30% up on the same period last year.
But like any change in approach, not everyone who has moved to online testing during the pandemic always finds it easy. Success, as always, lies in clear objectives and good planning.
We have been in the industry 30 years having been the first-ever provider of Internet testing in 1995. We know the pitfalls and we’ve learned to overcome them.
Here are my ten tips for managing the transition and making online assessment a success.
- Plan well. However much of a hurry you are in, a little bit of preparation will be worth it. Map out what your tests will cover and how they will work. Plan how to write questions, how to put them into assessments, how to pilot, how to deliver and how to report, and what to do in the event of things going awry. As Abraham Lincoln famously said: “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”
- Deliver from the Cloud. Unless you have a large, dedicated IT team, the days of delivering from on premise (behind the firewall) servers are over. Software-as-a-service platforms that run in the Cloud are scalable, secure and reliable. With test takers spread out and taking tests remotely, it’s the most practical approach.
- Use a provider that can deal with connectivity losses. Computers fail. Internet connections go down. Users make mistakes. Ensure that test-taker answers are stored safely so that if a failure happens, the answers made so far are saved and the test taker can resume. Questionmark has a “Save as you Go” capability which stores answers and allows resuming tests in the event of a failure. Other quality providers provide something similar.
- Pilot. Run a real trial with real test takers on a small scale before you run it on a large scale. For example, you may discover technical glitches that only arise for some devices, or other things that happen in the real world but not when testing internally. If you can’t do a big pilot, do a small one, you won’t regret it.
- If you can, stagger start times. Modern testing systems can cope with volume reasonably well, but it’s usually sensible to not have 1000s of test takers all start the test at the same time. Have a large item bank and allow testing “on demand” (at all times). Alternatively, if you have to have lots of tests on the same day, spread the load to have slight variations in start time.
- Work in partnership with test takers. Don’t think of testing as a confrontation between the test sponsor and test taker. The majority of test takers want to demonstrate their capability and to be assessed fairly. Give them full and transparent information about the testing process and they will be more likely to take the test honestly, and less likely to complain if things go awry.
- Also work in partnership with your test delivery company. Testing service companies like Questionmark want to provide the best service we can. We help our customers meet their goals. Listen to your vendor and help them listen and learn from you.
- Use a secure platform. Every time someone cheats at a test it devalues the results for all test takers. Use a platform that itself is secure (e.g. has ISO 27001 or similar security certification). Implement secure measures throughout the assessment lifecycle (e.g. authoring as well as delivery). Use appropriate security measures (e.g. secure browsers or remote proctoring), depending on the risks you face and how high the stakes of your tests. Although some organizations do deliver tests on learning management systems (LMSs), LMSs are typically not designed to do so securely, robustly, scalable and in a way that will make the tests valid and reliable.
- Be proportional in your balance between security and privacy. You want to discourage test takers from cheating but you also need to respect their privacy, especially if they are taking tests at home and you are using video to proctor them. For more information on this, see my post on Remote Proctoring & Privacy or else CNIL’s (the French data protection authority’s) recent guidance (in French but Google translate gives a gist) on monitoring online exams.
- Validity, reliability, fairness. Perhaps most importantly, remember what really matters and why you are doing it in the first place. All tests should aspire to be valid (measure what they are intended to), reliable (measure it consistently) and be fair (don’t disadvantage test takers for reasons irrelevant to what the test is measuring).
I hope this advice will help those starting online testing. As always, please reach out to me to continue the conversation.