Posted by John Kleeman
Assessment security has been an important topic at Questionmark, and that was echoed at the Questionmark Users Conference in Napa last week. Here is some advice I heard from attendees:
- Tip 1: It’s common to include an agreement page at the start of the assessment, where the participant agrees to follow the assessment rules, to keep exam content confidential and not to cheat. This discourages cheating by reducing people’s ability to rationalize that it’s okay to do so and also removes the potential for someone to claim they didn’t know the rules.
- Tip 2: It’s a good idea to have a formal agreement with SMEs in your organization who author or review questions to remind them not to pass the questions to others. If they are employees, you should involve your HR and legal departments in drafting the agreements or notices. That way if someone leaks content, you have HR and legal on board to deal with the disciplinary consequences.
- Tip 3: It’s prudent to use the capabilities of Questionmark software to restrict access to the item bank by topic. Only give authors access to the parts they are working on, to avoid inadvertent or deliberate content leakage.
- Tip 4: There is increasing interest and practical application of hybrid testing strategies for proctored and unprotected tests to allow you to focus anti-cheating resources on risk. For example, you might screen participants with quizzes, then give un-proctored tests and give those who pass a proctored test. Or you might deliver a series of short exams, at periodic intervals to make it harder for people to get proxy test takers to impersonate them. There is also a lot of interest in online proctoring, where people can take exams at home or in the office, and be proctored by a remote proctor using video monitoring. This reduces travel time and is often more secure than face-to-face proctoring.
- Tip 5: If your assessment system is on premise (behind the firewall), check regularly with your IT department that they are providing the security you need and that they are backing up your data. Most internal IT departments are hugely competent, but there is a risk as people change jobs over time that your IT department might lose touch with what the assessment application is used for. One user shared how their IT system failed to make backups of the Questionmark database, so when the server failed, they lost their data and had to restart from scratch. I’m sure this particular issue won’t happen for others, but IT teams have a wide set of priorities, so it’s good to check in with them.
There was lots more at the conference – iPads becoming mainstream for authoring and administration as well as delivery, people using OData to get access to Questionmark data, Questionmark being used to test the knowledge of soccer referees and some good thinking on balancing questions at higher cognitive levels.
One thing that particularly interested me was anecdotal evidence that having an internal employee certification program reduces employee attrition. Employees are less likely to leave your organization if you have an assessment and certification program. Certification makes employees feel more valued and more satisfied and so less likely to leave for a new job elsewhere. A couple of attendees shared that their internal statistics showed this.
This mirrors external research I’ve seen – for example the Aberdeen Group have published research which suggests that best-in-class organizations use assessments around twice as often as laggard organizations, and that first-year retention for best-of-class organizations is around 89% vs 76% for laggards.
For more information on security, download the white paper: Delivering Assessments Safely and Securely.