Posted by: John Kleeman, Founder and Executive Director
Back in 2015, when online proctoring was starting to get widely used, I wrote a couple of articles on the Questionmark blog:
- Online or test center proctoring: Which is best?
- Online or test center proctoring: Which is more secure?
With the current rapid increase in use of online proctoring caused by Covid-19, these articles are now two of the most popular on our blog. The first article remains a good summary of the pros and cons of online vs test center proctoring. But things have moved on in expectations around security in the last 5 years, so here is a 2020 perspective.
Privacy vs Security vs Candidate Experience
There are lots of things that can be done to improve the security of online proctoring, but they need to be balanced against candidate privacy and candidate satisfaction. Some (e.g., automatic facial recognition of candidates) run up against privacy laws. And some other measures make the test more secure but reduce candidate satisfaction.
Test publishers want their exams to be accessible, they want them to be secure and they must respect privacy laws like the GDPR and the Illinois Biometric Information Privacy act. These are opposing forces – you want to rigorously authenticate and scrutinize candidates, but you also want candidates to be able and willing to take the exam in volume. Security is critical for validity, but if candidates cannot take their exams because the security protocols are unwieldy, there must be a balance.
Here are some online proctoring measures which improve security, but which impact the experience or privacy of a candidate taking the test at home:
- Do you video the whole room or just the desk area?
- How specific are the requirements for the kind of computer/device used?
- Do you require a high level of bandwidth during the test to maintain security and then stop the test if bandwidth drops?
- Do you abandon the test if the candidate’s child or housemate comes into the testing room?
- Do you retain video recordings or other test data for a long duration to discourage use of proxy test takers who impersonate the candidate?
- Do you install software on the candidate device which tracks their activity or looks at other programs they are running to detect or prevent possible cheating?
- Do you use keystroke behavioral analysis or “AI” video analysis to try to detect cheating and cancel the test part way through if there are anomalies?
If you do all you can do for security, your program may be more secure, but you will encounter privacy legal issues in some jurisdictions and may reduce candidate satisfaction.
So what can we say about security?
Here are six things we can say most of which experts will broadly agree:
- Be holistic, there is more to security than just proctoring. Please look at all the risks facing your testing programme, not just proctoring, and then put in a comprehensive set of security measures to deal with mitigating these risks throughout the authoring, delivery and reporting process. See our recorded webinar 9 Risks to Test Security (and what to do about them) for an introduction to how to do this.
- Use “AI” as a flag for human input only. Maybe if I rewrite this article in 5 years’ time, AI proctoring will be good enough to identify cheating without human review, but in 2020 it can flag anomalies but not be relied on to do so fairly and to detect all integrity lapses. Use automation and AI to flag anomalies for human review, not to make decisions.
- In general, test centers give slightly better security than online proctoring. For most programs, there is good enough security in both modes, but it is slightly easier for an in-person proctor to check ID and see incidents of cheating. There is also better control of the computer used to take the test in a test center. Test sponsors use online proctoring because it saves the candidate travel time and is much more practical. Online proctoring is good enough security for most programs but test centers at least in theory have slightly better security.
- Beware unethical proctors (especially at test centers). Most proctors are very ethical, but there is a risk of an unethical proctor helping a candidate. This is more common at test centers where there is physical proximity than online where the proctor and candidate are remote, and all communication is recorded.
My 2015 article gave several public examples of corrupt proctors, and there continue to be reports of unethical test center staff aiding candidates. For example in 2019, the UK BBC discovered UK test centers charging money to help people pass minicab driving tests and also helping people pass immigration tests to help them stay in the UK. In the US, there was a lot of publicity last year for the college admissions scandal which included test center administrators being bribed to take the exam for students. These challenges are not just in the US and the UK, here is a recent report of corrupt test administration in India.
- Use continuous human review in online proctoring if concerned about content theft. There are two main kinds of online proctoring – one where a human observes the candidate throughout the test and one, often called “record and review” where the exam session is recorded but where a human proctor isn’t involved during the test but reviews afterwards. If you’re concerned about someone paying to take the test just to copy test content to sell it to others, then you need a live human proctor overseeing the test, as it will be too late to see it after the event.
- Consider whether frequent testing improves security. There is a strong argument that more frequent tests reduce peoples’ propensity to cheat. When you take tests frequently, it’s more hassle to organize a proxy test taker or otherwise work out how to cheat, and it may be simpler just to learn the material. More frequent testing also improves reliability by replacing one big measurement event with several. It’s obviously much easier to test frequently if you are proctoring remotely than if candidates have to travel to a test center.
Prior to Covid-19, there was a trend from test centers to online proctoring. Covid-19 has hugely accelerated this with many more programs using online proctoring and the change is here to stay. If you are considering online proctoring, the key issue is to strike the right balance between candidate experience, privacy and security.
If you’re interested in learning more about online proctoring, you can book a free a demo, or come to our webinar “An Ounce of Prevention: Proctoring and Test Security in 2020”. The webinar is on July 14th and you can register here.